-- Logs begin at Sun 2022-10-16 00:10:34 UTC, end at Sun 2022-10-16 01:14:27 UTC. -- Oct 16 00:48:28.354226 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 16 00:48:28.364616 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 16 00:48:28 2022] *** Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: nodename: dv-cinder-master-3-iscsi Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: machine: x86_64 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: clock source: unix Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: pcre jit disabled Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: detected number of CPU cores: 4 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: current working directory: / Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: detected binary path: /usr/bin/uwsgi-core Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: your processes number limit is 47680 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: your memory page size is 4096 bytes Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: detected max file descriptor number: 2048 Oct 16 00:48:28.367478 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: lock engine: pthread robust mutexes Oct 16 00:48:28.371063 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: thunder lock: enabled Oct 16 00:48:28.371063 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 16 00:48:28.371063 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 16 00:48:28.416657 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: Python main interpreter initialized at 0x5561e6fd49c0 Oct 16 00:48:28.416657 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: python threads support enabled Oct 16 00:48:28.416657 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: your server socket listen backlog is limited to 100 connections Oct 16 00:48:28.416657 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: your mercy for graceful operations on workers is 90 seconds Oct 16 00:48:28.417088 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: mapped 403077 bytes (393 KB) for 2 cores Oct 16 00:48:28.417088 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: *** Operational MODE: preforking *** Oct 16 00:48:28.417373 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: *** uWSGI is running in multiple interpreter mode *** Oct 16 00:48:28.417373 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: spawned uWSGI master process (pid: 103762) Oct 16 00:48:28.417587 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 16 00:48:28.418991 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: spawned uWSGI worker 1 (pid: 103763, cores: 1) Oct 16 00:48:28.418991 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: spawned uWSGI worker 2 (pid: 103764, cores: 1) Oct 16 00:48:28.418991 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 16 00:48:29.145963 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 16 00:48:29.148992 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 16 00:48:30.243048 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:48:30.243048 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: from cryptography.utils import int_from_bytes Oct 16 00:48:30.244163 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:48:30.244163 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: from cryptography.utils import int_from_bytes Oct 16 00:48:30.244877 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:48:30.244877 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: from cryptography.utils import int_from_bytes Oct 16 00:48:30.245300 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:48:30.245300 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: from cryptography.utils import int_from_bytes Oct 16 00:48:32.780697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 16 00:48:32.781281 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 16 00:48:32.781281 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 16 00:48:32.781740 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 16 00:48:32.781740 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 16 00:48:32.781905 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 16 00:48:32.781989 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 16 00:48:32.782095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 16 00:48:32.782261 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 16 00:48:32.782379 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 16 00:48:32.782379 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.782541 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.782694 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.782786 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783040 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783137 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783354 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783450 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783681 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783800 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.783991 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784134 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784308 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784407 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784624 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784714 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.784956 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785088 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785213 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785332 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785518 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785607 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785838 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.785928 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786152 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786242 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786455 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786554 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786761 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.786901 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787109 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787199 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787407 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787513 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787722 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.787811 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.788024 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.788130 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.788528 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.788662 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.788987 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789120 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789303 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789428 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789596 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789734 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.789937 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790070 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790250 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790372 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790558 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790660 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790890 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.790981 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791199 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791287 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791523 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791612 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791826 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.791917 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792234 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792331 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792558 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792667 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792864 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.792962 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.793171 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.793277 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794156 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794283 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794459 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794555 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794763 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.794940 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795106 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795195 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795431 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795529 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795736 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.795836 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796058 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796146 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796372 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796464 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796687 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796788 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.796999 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797086 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797298 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797416 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797597 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797695 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797899 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.797997 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798212 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798308 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798509 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798598 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798807 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.798962 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799071 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799155 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799371 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799489 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.799800 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800000 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800092 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800320 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800425 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800568 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800683 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800824 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.800933 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801118 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801214 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801431 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801522 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801733 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.801822 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802033 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802121 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802333 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802421 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802632 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802721 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.802960 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803080 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803260 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803363 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803562 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803650 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803861 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.803947 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804181 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804365 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804489 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804590 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804805 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.804903 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805105 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805206 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805416 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805505 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805819 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.805908 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806064 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806162 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806367 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806461 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806704 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.806792 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807044 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807150 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807351 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807441 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807664 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807760 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.807980 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808070 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808282 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808370 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808584 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808710 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.808918 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809017 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809334 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809574 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809678 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.809940 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810039 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810189 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810300 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810539 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810684 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810780 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.810921 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811121 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811240 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811424 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811528 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811727 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.811818 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812185 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812399 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812496 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812700 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.812787 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813002 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813091 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813304 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813447 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813607 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813756 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.813907 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814061 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814207 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814364 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814505 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814666 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.814805 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815000 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815138 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815310 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815437 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815611 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.815937 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816079 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816360 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816458 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816663 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816821 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.816961 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817051 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817264 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817363 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817579 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817675 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817878 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.817967 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818182 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818280 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818500 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818590 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818803 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.818992 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819127 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819219 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819443 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819532 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819748 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.819883 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820050 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820150 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820352 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820448 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820646 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820754 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.820947 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821052 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821247 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821335 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821548 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821647 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821851 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.821949 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822148 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822237 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822446 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822543 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822743 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.822839 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823068 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823160 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823367 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823492 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823668 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823805 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.823963 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.824109 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.824281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.824406 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.824645 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.824758 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825066 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825161 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825373 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825463 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825673 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.825975 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.826064 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.826297 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:48:32.826387 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.826627 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.826714 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827228 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827341 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827549 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827637 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827877 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.827966 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828207 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828303 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828523 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828601 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828831 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.828919 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829137 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829233 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829452 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829548 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829754 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.829908 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830060 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830162 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830452 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830671 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.830819 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831068 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831169 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831314 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831402 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831624 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831712 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.831937 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832035 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832244 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832332 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832547 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832635 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832851 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.832940 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833295 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833403 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833487 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833654 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833757 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.833841 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834089 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834179 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834429 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834519 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834739 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.834828 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835088 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835179 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835412 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835512 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835720 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.835811 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836026 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836115 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836329 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836636 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836741 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.836961 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837050 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837272 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837574 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837701 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837877 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.837966 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838182 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838285 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838494 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838593 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838798 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.838909 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839132 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839291 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839438 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839526 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839740 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.839827 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.840043 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.840669 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.840895 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841034 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841206 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841308 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841518 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841616 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.841998 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842097 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842180 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842261 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842425 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842515 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842730 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.842819 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843058 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843146 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843362 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843495 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843660 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843760 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.843952 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844054 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844248 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844345 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844544 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844632 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844840 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.844937 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845157 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845246 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845457 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845549 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845778 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.845884 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846076 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846163 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846384 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846473 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846709 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.846840 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847014 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847117 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847311 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847400 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847612 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847746 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847910 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.847999 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848209 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848299 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848505 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848595 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848825 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.848914 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849120 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849215 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849419 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849509 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849719 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.849807 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850017 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850113 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850316 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850405 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850618 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850713 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.850945 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851051 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851250 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851552 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851640 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851854 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.851953 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852166 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852256 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852461 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852550 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852764 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.852868 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853184 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853392 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853500 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853713 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.853841 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854026 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854127 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854334 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854423 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854632 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854724 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.854958 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.855062 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.855255 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.855344 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.855553 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.855642 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.856450 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.856596 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.856754 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.856921 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857052 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857193 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857349 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857499 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857645 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857807 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.857931 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858103 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858226 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858406 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858541 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858704 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.858847 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859052 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859164 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859360 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859450 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859661 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859765 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.859961 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860050 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860261 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860599 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860740 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860900 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.860994 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861240 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861332 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861573 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861664 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861881 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.861968 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862191 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862289 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862496 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862586 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862802 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.862913 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863129 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863233 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863434 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863524 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863744 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.863905 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864042 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864140 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864353 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864450 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864657 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864767 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.864966 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865064 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865283 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865375 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865579 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865669 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865876 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.865979 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866184 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866275 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866482 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866578 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866794 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.866969 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867146 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867234 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867462 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867563 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867768 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.867858 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868076 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868166 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868378 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868466 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868686 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868783 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.868988 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869076 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869291 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869379 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869596 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869739 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.869917 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870026 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870210 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870308 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870511 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870604 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870824 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.870931 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871148 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871239 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871487 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871595 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871785 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.871967 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.872093 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.872416 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.872739 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.873041 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.873345 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.873524 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.873657 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.873964 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874053 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874267 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874356 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874572 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874660 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874887 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.874980 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875189 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875277 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875493 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875595 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875803 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.875891 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876108 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876206 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876413 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876512 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876713 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.876810 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877019 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877107 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877321 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877427 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877614 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877702 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877910 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.877998 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878214 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878313 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878517 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878605 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878817 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.878923 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879151 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879243 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879454 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879543 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879751 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.879838 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880047 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880138 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880355 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880442 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880656 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880746 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.880956 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881046 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881255 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881344 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881551 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881639 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881870 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.881958 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882173 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882274 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882466 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882563 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882775 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.882926 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883100 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883201 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883402 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883499 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883700 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.883795 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884012 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884158 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884309 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884488 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884621 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884718 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.884929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885025 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885233 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885331 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885539 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885629 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885843 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.885932 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886147 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886246 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886450 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886538 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.886852 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887118 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887293 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887427 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887610 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887699 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.887922 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888074 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888229 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888320 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888538 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888627 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.888846 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889155 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889244 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889460 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889548 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889769 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.889867 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890076 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890165 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890375 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890471 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890682 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.890781 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891021 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891135 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891350 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891588 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891677 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.891919 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892021 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892233 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892352 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892531 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892630 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892830 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.892943 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893134 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893234 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893432 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893523 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893731 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.893820 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894029 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894119 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894331 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894420 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894626 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894715 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.894943 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895049 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895245 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895334 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895543 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895632 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895853 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.895944 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896148 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896238 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896448 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896549 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896743 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.896831 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897045 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897134 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897345 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897443 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897639 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897727 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.897937 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898025 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898235 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898332 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898534 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898623 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898841 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.898990 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899170 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899275 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899490 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899581 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899795 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.899891 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900186 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900397 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900498 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900695 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.900828 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901032 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901124 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901333 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901422 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901630 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901725 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.901934 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.902032 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.902236 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.902339 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.902534 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903266 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903371 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903502 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903675 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903765 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.903980 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904085 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904282 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904579 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904676 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904882 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.904981 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905179 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905269 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905486 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905576 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905788 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.905886 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906084 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906172 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906379 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906481 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906682 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.906781 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907004 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907108 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907303 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907392 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907607 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907697 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907902 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.907991 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908204 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908292 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908500 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908590 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908795 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.908884 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909095 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909183 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909396 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909485 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909697 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909787 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.909989 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910076 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910290 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910389 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910593 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910681 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.910908 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911013 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911209 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911300 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911506 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911595 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911801 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.911897 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912098 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912187 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912400 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912489 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912717 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.912830 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913025 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913115 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913326 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913425 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913628 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913729 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.913926 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914016 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914223 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914311 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914524 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914651 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914821 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.914999 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915138 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915323 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915427 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915640 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915729 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.915942 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916032 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916249 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916348 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916551 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916642 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916855 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.916944 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917158 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917256 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917454 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917543 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917752 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.917841 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.918090 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.918181 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.918449 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.918553 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.918801 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919013 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919159 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919249 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919420 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919510 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919736 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.919825 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920036 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920125 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920339 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920427 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920638 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920733 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.920933 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921029 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921245 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921334 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921545 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921643 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921850 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.921940 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922149 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922238 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922445 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922551 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.922908 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923092 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923194 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923396 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923516 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923696 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.923784 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924008 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924097 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924315 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924415 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924615 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924715 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.924915 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925006 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925211 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925338 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925511 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925600 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925813 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.925911 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926114 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926205 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926420 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926509 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926725 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.926815 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927051 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927144 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927370 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927470 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927659 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927748 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.927958 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928047 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928258 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928346 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928560 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928648 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928863 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.928960 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929168 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929273 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929469 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929568 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929767 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.929857 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930067 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930156 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930365 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930463 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930664 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930753 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.930981 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931126 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931284 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931413 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931582 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931717 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.931900 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932021 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932204 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932302 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932504 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932592 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932824 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.932961 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.933105 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.933196 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.933410 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.933503 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.933708 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.934312 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.934669 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.934912 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935006 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935089 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935294 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935384 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935596 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935686 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935895 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.935985 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936193 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936281 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936495 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936587 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936805 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.936892 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937100 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937266 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937398 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937509 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937694 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.937822 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938009 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938130 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938307 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938461 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938620 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.938710 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939031 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939141 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939338 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939428 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939633 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939720 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.939943 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940039 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940240 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940337 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940537 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940627 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940853 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.940943 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941154 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941247 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941470 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941559 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941800 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.941896 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.942108 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.942207 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.942412 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.942501 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.942991 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943092 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943189 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943273 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943355 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943463 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943651 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943779 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.943950 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944096 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944264 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944415 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944578 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944716 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.944886 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945024 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945200 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945329 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945605 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945704 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.945924 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946014 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946229 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946327 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946538 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946635 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946836 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.946996 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947163 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947253 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947461 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947629 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.947854 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948062 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948158 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948360 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948464 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948675 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948779 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.948972 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949062 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949282 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949658 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949775 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.949955 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950046 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950330 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950421 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950640 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950730 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.950964 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951058 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951268 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951367 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951583 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951672 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951883 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.951971 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.952443 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.952534 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.952749 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.952847 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953046 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953207 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953346 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953435 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953645 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953773 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.953941 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954038 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954240 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954329 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954536 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954625 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.954937 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955055 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955186 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955284 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955500 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955590 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955802 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.955900 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956123 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956212 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956425 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956512 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956727 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.956829 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957024 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957124 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957325 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957425 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957658 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957747 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.957963 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958107 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958378 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958581 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958672 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958896 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.958990 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959212 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959313 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959513 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959603 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.959901 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960113 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960215 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960417 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960507 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960714 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.960842 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961012 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961115 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961312 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961450 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961628 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961718 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.961927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962030 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962224 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962319 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962524 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962614 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962823 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.962993 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963149 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963446 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963534 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963743 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.963832 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964039 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964138 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964356 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964455 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964654 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964744 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.964976 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.965613 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.965803 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.965894 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966102 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966193 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966475 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966564 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966808 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.966947 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967143 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967241 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967454 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967551 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967764 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.967866 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968070 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968159 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968382 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968473 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968692 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.968790 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969001 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969090 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969314 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969562 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969652 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969890 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.969981 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970198 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970290 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970506 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970605 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970808 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.970926 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971137 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971227 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971445 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971544 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971749 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.971839 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972053 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972304 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972401 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972628 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972722 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.972938 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973035 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973249 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973406 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973553 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973645 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973861 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.973951 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974169 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974268 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974477 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974565 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974780 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.974908 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975112 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975234 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975415 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975506 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975745 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.975897 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976046 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976137 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976356 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976446 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976666 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976755 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.976976 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977068 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977284 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977379 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977586 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977675 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977895 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.977995 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978202 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978299 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978510 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978653 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978811 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.978923 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979159 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979247 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979466 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979579 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979775 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.979864 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980082 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980173 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980390 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980481 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980732 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.980850 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.981131 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.981222 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.981511 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.981643 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.981994 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982086 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982312 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982403 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982625 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982716 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.982973 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983068 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983306 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983399 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983620 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983710 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.983930 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984026 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984243 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984350 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984542 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984630 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.984842 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985077 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985167 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985439 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985535 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985738 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.985827 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986055 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986145 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986362 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986453 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986661 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.986758 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987007 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987119 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987312 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987402 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987670 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987843 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.987977 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988194 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988285 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988511 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988599 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988836 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.988925 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989158 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989249 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989472 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989561 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989772 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.989860 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990089 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990180 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990486 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990693 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.990935 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991030 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991282 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991373 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991596 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991682 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991909 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.991998 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992216 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992314 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992520 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992611 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992820 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.992910 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993124 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993422 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993511 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993721 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.993810 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994029 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994116 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994329 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994431 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994626 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994714 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.994948 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995040 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995247 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995348 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995546 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995636 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995844 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.995940 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.996139 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.996236 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.996962 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.997055 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.997364 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.997453 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.997729 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.997828 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998034 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998123 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998335 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998424 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998649 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998748 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.998976 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999070 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999278 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999380 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999581 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999672 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999881 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:32.999971 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000189 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000279 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000488 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000578 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000794 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.000884 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001093 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001186 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001400 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001491 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001703 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.001837 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002040 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002138 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002339 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002429 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002654 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002879 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.002972 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003216 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003307 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003531 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003858 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.003946 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004176 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004267 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004478 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004566 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004779 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.004869 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005081 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005180 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005399 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005488 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005705 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.005940 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006038 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006269 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006584 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006673 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.006903 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007008 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007210 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007301 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007514 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007611 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007819 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.007910 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008119 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008215 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008419 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008507 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008722 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.008812 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009024 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009113 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009326 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009424 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009627 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009717 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.009930 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010019 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010235 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010335 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010540 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010629 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010844 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.010973 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011169 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011272 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011473 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011562 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011773 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.011860 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012138 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012240 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012545 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012635 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012871 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.012959 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013183 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013273 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013507 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013605 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013816 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.013906 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.014120 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.014210 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.014482 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.014796 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.014913 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015129 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015222 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015435 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015524 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015750 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.015839 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016056 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016146 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016373 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016462 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016692 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016789 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.016999 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.017090 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.017314 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.017404 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.017621 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.017944 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018032 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018276 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018373 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018598 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018688 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.018941 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019046 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019244 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019334 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019542 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019632 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019840 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.019931 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020146 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020235 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020459 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020558 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020763 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.020854 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021061 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021152 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021357 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021456 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021658 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021748 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.021956 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022045 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022253 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022351 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022550 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022639 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022847 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.022949 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023170 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023272 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023466 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023556 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023762 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.023851 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024058 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024147 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024350 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024445 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024645 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024742 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.024939 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025029 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025236 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025324 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025531 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025824 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.025920 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026117 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026205 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026409 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026496 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026702 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.026791 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027020 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027112 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027317 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027417 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027652 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.027745 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028061 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028151 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028436 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028551 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028689 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028774 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.028964 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029069 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029264 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029364 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029562 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029651 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029859 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.029955 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.030173 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.030272 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.030499 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.030596 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.030850 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031032 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031139 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031223 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031423 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031520 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031725 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.031815 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032029 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032119 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032328 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032416 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032647 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032736 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.032958 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033062 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033255 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033343 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033571 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033668 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033874 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.033963 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034181 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034270 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034485 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034629 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034789 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.034913 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035119 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035219 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035425 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035588 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035729 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.035819 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036032 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036121 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036336 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036426 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036638 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036734 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.036946 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037045 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037248 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037346 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037551 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037641 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037855 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.037945 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038161 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038251 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038477 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038620 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038783 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.038903 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039106 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039196 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039409 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039497 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039715 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.039818 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040026 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040117 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040330 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040421 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040634 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040734 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.040945 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041043 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041254 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041353 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041569 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041666 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041869 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.041958 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.042107 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103764) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 16 00:48:33.042197 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.042488 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.042810 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.043690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.044169 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.044324 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.044647 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.044958 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.045266 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.045566 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:48:33.045803 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=103763) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 16 00:48:33.195460 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO dbcounter [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Registered counter for database nova_api Oct 16 00:48:33.198035 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO dbcounter [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Registered counter for database nova_api Oct 16 00:48:33.218092 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_db.sqlalchemy.engines [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103764) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.218967 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_db.sqlalchemy.engines [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103763) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.225578 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writer thread running {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.226814 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writer thread running {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.283801 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.context [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d19e9fc7-2682-4022-903d-afef81878fc4(cell1) {{(pid=103763) load_cells /opt/stack/nova/nova/context.py:464}} Oct 16 00:48:33.284008 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.context [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d19e9fc7-2682-4022-903d-afef81878fc4(cell1) {{(pid=103764) load_cells /opt/stack/nova/nova/context.py:464}} Oct 16 00:48:33.287640 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:48:33.287764 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:48:33.288114 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:48:33.288271 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:48:33.289664 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:48:33.289779 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:48:33.290377 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:48:33.290481 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:48:33.290816 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:48:33.290957 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:48:33.292822 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_concurrency.lockutils [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103764) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:48:33.292941 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:48:33.295357 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO dbcounter [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Registered counter for database nova_cell0 Oct 16 00:48:33.295492 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO dbcounter [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Registered counter for database nova_cell0 Oct 16 00:48:33.299427 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO dbcounter [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Registered counter for database nova_cell1 Oct 16 00:48:33.299556 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO dbcounter [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Registered counter for database nova_cell1 Oct 16 00:48:33.307271 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_db.sqlalchemy.engines [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103763) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.308741 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_db.sqlalchemy.engines [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103764) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.311369 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_db.sqlalchemy.engines [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103764) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.312670 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writer thread running {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.313357 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_db.sqlalchemy.engines [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103763) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.316819 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writer thread running {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.319076 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writer thread running {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.457356 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writer thread running {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.466184 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO dbcounter [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Registered counter for database nova_cell0 Oct 16 00:48:33.467256 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO dbcounter [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Registered counter for database nova_cell0 Oct 16 00:48:33.475184 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_db.sqlalchemy.engines [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103763) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.479225 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG oslo_db.sqlalchemy.engines [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103764) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:48:33.480018 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writer thread running {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.483872 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writer thread running {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:48:33.822535 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: WARNING keystonemiddleware.auth_token [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:48:33.825404 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: WARNING keystonemiddleware.auth_token [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:48:33.831428 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: WARNING keystonemiddleware.auth_token [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:48:33.833077 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: WARNING keystonemiddleware.auth_token [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:48:34.176124 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: WARNING keystonemiddleware.auth_token [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:48:34.178542 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: WARNING keystonemiddleware.auth_token [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:48:34.181964 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: WARNING keystonemiddleware.auth_token [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:48:34.185095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: WARNING keystonemiddleware.auth_token [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:48:34.187958 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5561e6fd49c0 pid: 103764 (default app) Oct 16 00:48:34.190095 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:48:34.190462 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5561e6fd49c0 pid: 103763 (default app) Oct 16 00:48:34.191310 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-0d49c790-e071-49ed-a3c9-c0328bac2bd7 None None] 10.222.0.25 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001829 Oct 16 00:48:34.191719 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 788 bytes} [Sun Oct 16 00:48:34 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:01.424409 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:01.426557 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002989 Oct 16 00:50:01.428298 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:01 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:01.440408 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_api:SELECT=3 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:01.441364 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_cell0:SELECT=3 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:01.443762 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_cell1:SELECT=3 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:01.444693 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:02.479005 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-50de522f-2476-4f75-b2ba-e80a8e4a3f7f admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:02.513175 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-50de522f-2476-4f75-b2ba-e80a8e4a3f7f admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.078259 Oct 16 00:50:02.519040 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 2/3] 10.222.0.25 () {60 vars in 1292 bytes} [Sun Oct 16 00:50:01 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1085 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:05.251301 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:05.252340 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-61ed8b9a-7e54-4ced-970a-84a93ea44162 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Oct 16 00:50:05.258757 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 2/4] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:05 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:05.896843 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-8f6d4e6d-13ae-43b3-9107-a6a869636df3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "swap": 0, "rxtx_factor": 1.0, "id": "c1", "name": "cirros256", "ram": 256, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 1}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:05.954747 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-8f6d4e6d-13ae-43b3-9107-a6a869636df3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.694579 Oct 16 00:50:05.955640 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 3/5] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:50:05.971774 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=3 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:05.972936 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_cell0:SELECT=3 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:05.975300 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_cell1:SELECT=3 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:05.976746 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:06.191934 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-66c7043a-9bcc-4ebb-91e6-59b95e63375c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:06.273163 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-66c7043a-9bcc-4ebb-91e6-59b95e63375c admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.306749 Oct 16 00:50:06.279012 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 3/6] 10.222.0.25 () {64 vars in 1336 bytes} [Sun Oct 16 00:50:05 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:09.091324 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-8f6d4e6d-13ae-43b3-9107-a6a869636df3 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:09.093058 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-8f6d4e6d-13ae-43b3-9107-a6a869636df3 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002185 Oct 16 00:50:09.093488 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 4/7] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:09 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:10.042354 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-ce38e795-b0f4-41b2-92b9-f7ec74ebdf88 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "os-flavor-access:is_public": true, "name": "ds512M", "swap": 0, "disk": 5, "ram": 512, "id": "d1"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:10.097654 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-ce38e795-b0f4-41b2-92b9-f7ec74ebdf88 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.996706 Oct 16 00:50:10.107201 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 4/8] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:09 2022] POST /compute/v2.1/flavors => generated 404 bytes in 1003 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:50:10.117218 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-285adf54-ea3c-4ce8-b2ac-226e4041d32a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:10.161219 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-285adf54-ea3c-4ce8-b2ac-226e4041d32a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.050894 Oct 16 00:50:10.161808 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 5/9] 10.222.0.25 () {64 vars in 1336 bytes} [Sun Oct 16 00:50:10 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:12.907926 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-ce38e795-b0f4-41b2-92b9-f7ec74ebdf88 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:12.909938 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-ce38e795-b0f4-41b2-92b9-f7ec74ebdf88 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002838 Oct 16 00:50:12.910966 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:12 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:13.470417 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-ae079ca4-a2d9-4328-8839-76260e185833 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "name": "ds1G", "ram": 1024, "vcpus": 1, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:13.494393 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-ae079ca4-a2d9-4328-8839-76260e185833 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.572186 Oct 16 00:50:13.495123 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 6/11] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:12 2022] POST /compute/v2.1/flavors => generated 404 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:50:13.507730 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-d90243a6-f5a7-435c-a16e-032e0dc3122d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:13.543950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-d90243a6-f5a7-435c-a16e-032e0dc3122d admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039300 Oct 16 00:50:13.544283 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 6/12] 10.222.0.25 () {64 vars in 1336 bytes} [Sun Oct 16 00:50:13 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:16.310314 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-ae079ca4-a2d9-4328-8839-76260e185833 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:16.311357 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-ae079ca4-a2d9-4328-8839-76260e185833 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Oct 16 00:50:16.312318 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 7/13] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:16 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:16.895247 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-9c03389a-9bd5-4d8c-8505-fd197357719a admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 2, "id": "d3", "ram": 2048, "name": "ds2G", "os-flavor-access:is_public": true, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:16.917810 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-9c03389a-9bd5-4d8c-8505-fd197357719a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.599231 Oct 16 00:50:16.918655 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 7/14] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:16 2022] POST /compute/v2.1/flavors => generated 404 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:50:16.933107 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-d3ec4073-2b52-42ae-be45-05b2e4572e6f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:16.969713 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-d3ec4073-2b52-42ae-be45-05b2e4572e6f admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.042507 Oct 16 00:50:16.970578 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 8/15] 10.222.0.25 () {64 vars in 1336 bytes} [Sun Oct 16 00:50:16 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:19.753642 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-9c03389a-9bd5-4d8c-8505-fd197357719a admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:19.754774 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-9c03389a-9bd5-4d8c-8505-fd197357719a admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Oct 16 00:50:19.755251 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 8/16] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:20.356523 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 20, "swap": 0, "os-flavor-access:is_public": true, "id": "d4", "vcpus": 4, "rxtx_factor": 1.0, "ram": 4096}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:20.376552 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.614158 Oct 16 00:50:20.377110 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 9/17] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:19 2022] POST /compute/v2.1/flavors => generated 404 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:50:20.389549 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-c28e58ed-1ca4-4626-b84c-49aaa67bce19 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:20.416629 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-c28e58ed-1ca4-4626-b84c-49aaa67bce19 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030691 Oct 16 00:50:20.417233 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 9/18] 10.222.0.25 () {64 vars in 1336 bytes} [Sun Oct 16 00:50:20 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:23.175293 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:23.176506 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001627 Oct 16 00:50:23.177157 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 10/19] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:23 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:23.653351 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-19519c4c-2492-4cd2-98c1-c8a36b861cfd admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:23.678438 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-19519c4c-2492-4cd2-98c1-c8a36b861cfd admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.488993 Oct 16 00:50:23.679058 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 10/20] 10.222.0.25 () {60 vars in 1292 bytes} [Sun Oct 16 00:50:23 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:50:26.390300 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:26.391971 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-82f4a7d8-9b4e-47ff-bbcc-29d191a9c595 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002171 Oct 16 00:50:26.392536 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 11/21] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:26.890374 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-b215886d-3da8-494e-8d19-879c868e0e23 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "disk": 1, "ram": 512, "os-flavor-access:is_public": true, "id": "1", "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:26.917631 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-b215886d-3da8-494e-8d19-879c868e0e23 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.516802 Oct 16 00:50:26.918531 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 11/22] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:26 2022] POST /compute/v2.1/flavors => generated 402 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:50:26.935596 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-aa3ecbad-4e80-4c61-97c4-b914f9b9e244 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:26.972744 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-aa3ecbad-4e80-4c61-97c4-b914f9b9e244 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.041576 Oct 16 00:50:26.973424 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 12/23] 10.222.0.25 () {64 vars in 1333 bytes} [Sun Oct 16 00:50:26 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:29.907415 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-b215886d-3da8-494e-8d19-879c868e0e23 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:29.908615 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-b215886d-3da8-494e-8d19-879c868e0e23 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001741 Oct 16 00:50:29.909128 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:30.442228 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-1ec8d0b5-b8c9-4c1a-adf4-f78006115dff admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "id": "2", "vcpus": 1, "rxtx_factor": 1.0, "disk": 20, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "m1.small"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:30.466973 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-1ec8d0b5-b8c9-4c1a-adf4-f78006115dff admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.550165 Oct 16 00:50:30.467707 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 13/25] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:29 2022] POST /compute/v2.1/flavors => generated 405 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:50:30.486809 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-043339e2-ba11-49e7-99d1-265ec3b96e3b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:30.520855 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-043339e2-ba11-49e7-99d1-265ec3b96e3b admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038078 Oct 16 00:50:30.527148 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 13/26] 10.222.0.25 () {64 vars in 1333 bytes} [Sun Oct 16 00:50:30 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:33.407581 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-1ec8d0b5-b8c9-4c1a-adf4-f78006115dff admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:33.413602 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-1ec8d0b5-b8c9-4c1a-adf4-f78006115dff admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.007057 Oct 16 00:50:33.414357 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 14/27] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:33 2022] GET /compute/v2.1 => generated 388 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:50:33.928234 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-8b58d59a-7c38-4a29-9be4-16327aecfab5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "swap": 0, "id": "3", "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "m1.medium", "os-flavor-access:is_public": true, "ram": 4096}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:33.960303 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-8b58d59a-7c38-4a29-9be4-16327aecfab5 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.538754 Oct 16 00:50:33.961489 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 14/28] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:33 2022] POST /compute/v2.1/flavors => generated 406 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:50:33.975748 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-052fce0f-d9db-433f-8144-825e4a2cebca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:33.982150 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:34.011451 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-052fce0f-d9db-433f-8144-825e4a2cebca admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039447 Oct 16 00:50:34.012737 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 15/29] 10.222.0.25 () {64 vars in 1333 bytes} [Sun Oct 16 00:50:33 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:37.035044 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-8b58d59a-7c38-4a29-9be4-16327aecfab5 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:37.036982 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-8b58d59a-7c38-4a29-9be4-16327aecfab5 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002658 Oct 16 00:50:37.038070 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 15/30] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:37 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:37.667221 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-9bcbb82a-359d-4dde-950c-31e1c61df8f9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 80, "os-flavor-access:is_public": true, "id": "4", "ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "rxtx_factor": 1.0, "name": "m1.large"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:37.778726 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-9bcbb82a-359d-4dde-950c-31e1c61df8f9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.730150 Oct 16 00:50:37.781444 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 16/31] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:37 2022] POST /compute/v2.1/flavors => generated 405 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:50:37.797110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-31f8f948-0447-4560-9500-a2ab7b0e0cf7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:37.802951 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:37.879807 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-31f8f948-0447-4560-9500-a2ab7b0e0cf7 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.086310 Oct 16 00:50:37.882213 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 16/32] 10.222.0.25 () {64 vars in 1333 bytes} [Sun Oct 16 00:50:37 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:50:40.957112 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-9bcbb82a-359d-4dde-950c-31e1c61df8f9 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:40.959011 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-9bcbb82a-359d-4dde-950c-31e1c61df8f9 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002296 Oct 16 00:50:40.959416 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 17/33] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:50:41.594230 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 160, "rxtx_factor": 1.0, "id": "5", "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384, "swap": 0, "vcpus": 8}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:41.627070 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.660183 Oct 16 00:50:41.627772 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 17/34] 10.222.0.25 () {64 vars in 1283 bytes} [Sun Oct 16 00:50:40 2022] POST /compute/v2.1/flavors => generated 408 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:50:41.647464 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-443a175c-0f48-4344-a3e3-a1f6fc95e16e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:50:41.687739 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-443a175c-0f48-4344-a3e3-a1f6fc95e16e admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.045809 Oct 16 00:50:41.688660 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 18/35] 10.222.0.25 () {64 vars in 1333 bytes} [Sun Oct 16 00:50:41 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:50:59.083558 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:59.084661 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001730 Oct 16 00:50:59.085066 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 18/36] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:50:59 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:50:59.095834 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:50:59.488846 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-038fd365-2bfb-44d5-afd0-9578fe44a3b0 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:50:59.507336 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-038fd365-2bfb-44d5-afd0-9578fe44a3b0 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.415885 Oct 16 00:50:59.507873 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 19/37] 10.222.0.25 () {60 vars in 1292 bytes} [Sun Oct 16 00:50:59 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:51:00.876873 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:00.877897 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-2e662f8b-1f36-4d35-a305-e0c2622e28b3 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 Oct 16 00:51:00.878299 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:01.456564 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-8889f9dc-ad7c-46ad-9ed0-3d1c9b545eec admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:01.462965 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.wsgi [None req-8889f9dc-ad7c-46ad-9ed0-3d1c9b545eec admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-8889f9dc-ad7c-46ad-9ed0-3d1c9b545eec admin admin] Request method failure captured: Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept: application/json Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept-Encoding: gzip, deflate Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Connection: keep-alive Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Content-Length: 0 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Host: 10.222.0.25 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-System-Scope: None Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: User-Agent: python-novaclient Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Auth-Token: gAAAAABjS1V0S4U_WqAqbQPqrV2wh3k3lXfk78UZyzOO4l627VhSeZZrCsc_CZXxq5pkL2SXk2nW5KLcWTOktSP_3mRuwxkX7sEGD7RRHRHa9Vh_ckZESvECN1hg1R1hMdagEUFxR6CE990tKen4wYbQ6WfH2eR4ARQfbM7yIoH-r_DZXBu8dH4 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Id: None Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Name: None Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Identity-Status: Confirmed Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Is-Admin-Project: True Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Id: default Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Name: Default Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Name: admin Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Role: admin,reader,member Oct 16 00:51:01.466277 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Roles: admin,reader,member Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant: admin Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Name: admin Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User: admin Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Id: default Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Name: Default Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Name: admin Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: method: > Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: exception: Flavor 42 could not be found. Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: action_args: {'id': '42'} Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:01.467927 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:01.468812 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.469929 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-8889f9dc-ad7c-46ad-9ed0-3d1c9b545eec admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103764) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:01.471032 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-8889f9dc-ad7c-46ad-9ed0-3d1c9b545eec admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.446861 Oct 16 00:51:01.471032 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 20/39] 10.222.0.25 () {60 vars in 1190 bytes} [Sun Oct 16 00:51:01 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 448 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:01.478721 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-96668286-9ba8-4597-b706-30c86435ef2a admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:01.483066 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:51:01.504951 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-96668286-9ba8-4597-b706-30c86435ef2a admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.030696 Oct 16 00:51:01.505581 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 20/40] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:51:01.516832 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-a60f981c-2d92-479b-ba40-758f6d33f64b admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:01.541445 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-a60f981c-2d92-479b-ba40-758f6d33f64b admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.028584 Oct 16 00:51:01.541445 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 21/41] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:01 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:51:01.551060 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-8574a105-353c-4c64-b361-838a973ff311 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:01.557351 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.wsgi [None req-8574a105-353c-4c64-b361-838a973ff311 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-8574a105-353c-4c64-b361-838a973ff311 admin admin] Request method failure captured: Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept: application/json Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept-Encoding: gzip, deflate Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Connection: keep-alive Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Content-Length: 0 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Host: 10.222.0.25 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-System-Scope: None Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: User-Agent: python-novaclient Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Auth-Token: gAAAAABjS1V0S4U_WqAqbQPqrV2wh3k3lXfk78UZyzOO4l627VhSeZZrCsc_CZXxq5pkL2SXk2nW5KLcWTOktSP_3mRuwxkX7sEGD7RRHRHa9Vh_ckZESvECN1hg1R1hMdagEUFxR6CE990tKen4wYbQ6WfH2eR4ARQfbM7yIoH-r_DZXBu8dH4 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Id: None Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Name: None Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Identity-Status: Confirmed Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Is-Admin-Project: True Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Id: default Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Name: Default Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Name: admin Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Role: admin,reader,member Oct 16 00:51:01.560684 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Roles: admin,reader,member Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant: admin Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Name: admin Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User: admin Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Id: default Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Name: Default Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Name: admin Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: method: > Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: exception: Flavor 42 could not be found. Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: action_args: {'id': '42'} Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:01.562381 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:01.563322 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:01.564372 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-8574a105-353c-4c64-b361-838a973ff311 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=103763) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:01.565378 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-8574a105-353c-4c64-b361-838a973ff311 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014707 Oct 16 00:51:01.565378 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 21/42] 10.222.0.25 () {60 vars in 1190 bytes} [Sun Oct 16 00:51:01 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:02.805289 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-a60f981c-2d92-479b-ba40-758f6d33f64b admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:02.806457 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-a60f981c-2d92-479b-ba40-758f6d33f64b admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002025 Oct 16 00:51:02.806891 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 22/43] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:02 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:03.341047 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-91e76f95-f9f9-49f6-b79d-46dd316ff6c7 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:03.346095 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.wsgi [None req-91e76f95-f9f9-49f6-b79d-46dd316ff6c7 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-91e76f95-f9f9-49f6-b79d-46dd316ff6c7 admin admin] Request method failure captured: Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept: application/json Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept-Encoding: gzip, deflate Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Connection: keep-alive Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Content-Length: 0 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Host: 10.222.0.25 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-System-Scope: None Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: User-Agent: python-novaclient Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Auth-Token: gAAAAABjS1V2FBV8GzEUK5nBch1fVqlDam8247UwXe6hEnr_jT-aaGKFjrD1hjI6rWBW7Z_uRFYBMThicDDLBGm1hq7P5vvFlLjZSTi81S9JCJF4oBLhTj3lKMi-U4_xMgmo7YxLP6jCk-93FaNutlHhUVb_AllbKG6X8ZrRP4nUDeZKeU8YnMQ Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Id: None Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Name: None Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Identity-Status: Confirmed Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Is-Admin-Project: True Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Id: default Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Name: Default Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Name: admin Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Role: admin,reader,member Oct 16 00:51:03.347620 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Roles: admin,reader,member Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant: admin Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Name: admin Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User: admin Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Id: default Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Name: Default Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Name: admin Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: method: > Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: exception: Flavor 84 could not be found. Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: action_args: {'id': '84'} Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:03.349856 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:03.350950 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.352365 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-91e76f95-f9f9-49f6-b79d-46dd316ff6c7 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103763) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:03.353716 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-91e76f95-f9f9-49f6-b79d-46dd316ff6c7 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.403143 Oct 16 00:51:03.353716 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 22/44] 10.222.0.25 () {60 vars in 1190 bytes} [Sun Oct 16 00:51:02 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 405 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:03.359337 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-53e67154-9bc1-496e-a6d9-aacb13503158 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:03.374653 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-53e67154-9bc1-496e-a6d9-aacb13503158 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018864 Oct 16 00:51:03.375409 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 23/45] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:51:03.385070 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-e283d6ff-a270-4804-b9ae-a93fe345d1bb admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:03.400479 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-e283d6ff-a270-4804-b9ae-a93fe345d1bb admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019036 Oct 16 00:51:03.401023 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:03 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:51:03.410499 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-5d37eec3-147c-4fec-b137-c917f9cb0b01 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:03.416197 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.wsgi [None req-5d37eec3-147c-4fec-b137-c917f9cb0b01 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-5d37eec3-147c-4fec-b137-c917f9cb0b01 admin admin] Request method failure captured: Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept: application/json Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept-Encoding: gzip, deflate Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Connection: keep-alive Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Content-Length: 0 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Host: 10.222.0.25 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-System-Scope: None Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: User-Agent: python-novaclient Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Auth-Token: gAAAAABjS1V2FBV8GzEUK5nBch1fVqlDam8247UwXe6hEnr_jT-aaGKFjrD1hjI6rWBW7Z_uRFYBMThicDDLBGm1hq7P5vvFlLjZSTi81S9JCJF4oBLhTj3lKMi-U4_xMgmo7YxLP6jCk-93FaNutlHhUVb_AllbKG6X8ZrRP4nUDeZKeU8YnMQ Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Id: None Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Name: None Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Identity-Status: Confirmed Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Is-Admin-Project: True Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Id: default Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Name: Default Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Name: admin Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Role: admin,reader,member Oct 16 00:51:03.417697 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Roles: admin,reader,member Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant: admin Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Name: admin Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User: admin Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Id: default Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Name: Default Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Name: admin Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: method: > Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: exception: Flavor 84 could not be found. Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: action_args: {'id': '84'} Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:03.419287 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:03.420281 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:03.421390 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-5d37eec3-147c-4fec-b137-c917f9cb0b01 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=103764) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:03.422608 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-5d37eec3-147c-4fec-b137-c917f9cb0b01 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012122 Oct 16 00:51:03.422608 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 24/47] 10.222.0.25 () {60 vars in 1190 bytes} [Sun Oct 16 00:51:03 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:04.687003 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-e283d6ff-a270-4804-b9ae-a93fe345d1bb admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:04.688042 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-e283d6ff-a270-4804-b9ae-a93fe345d1bb admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Oct 16 00:51:04.688443 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 24/48] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:05.229836 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-e135c23c-8170-474f-8791-e6aaf267ae00 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:05.235931 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.wsgi [None req-e135c23c-8170-474f-8791-e6aaf267ae00 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-e135c23c-8170-474f-8791-e6aaf267ae00 admin admin] Request method failure captured: Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept: application/json Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Accept-Encoding: gzip, deflate Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Connection: keep-alive Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Content-Length: 0 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Host: 10.222.0.25 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Openstack-System-Scope: None Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: User-Agent: python-novaclient Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Auth-Token: gAAAAABjS1V4XGbgD1j15pBFZIMFb0MUmHUUjY2lMJK5FbO29AInMdf_RrnfFsoEiFUdvaCIFFCjW5xI673jVyyLqOsmzkOy97iu2dGB-nSWH_NVU92Uc0JSBeQKU-Hau5PY8E9_AzkpuPpJaDPrk7pVAgwZhtoBx7-gd5hXTL30LDdpYzU3WWQ Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Id: None Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Domain-Name: None Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Identity-Status: Confirmed Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Is-Admin-Project: True Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Id: default Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Domain-Name: Default Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Project-Name: admin Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Role: admin,reader,member Oct 16 00:51:05.237456 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Roles: admin,reader,member Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant: admin Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-Tenant-Name: admin Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User: admin Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Id: default Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Domain-Name: Default Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: X-User-Name: admin Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: method: > Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: exception: Flavor 451 could not be found. Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: action_args: {'id': '451'} Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:05.239238 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:05.241016 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.242359 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-e135c23c-8170-474f-8791-e6aaf267ae00 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103764) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:05.243806 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-e135c23c-8170-474f-8791-e6aaf267ae00 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.413171 Oct 16 00:51:05.243806 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 25/49] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:51:04 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 415 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:05.249721 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-07f9afdc-d48f-4d7e-af50-6f592f4ca472 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:05.265362 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-07f9afdc-d48f-4d7e-af50-6f592f4ca472 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019222 Oct 16 00:51:05.265933 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 25/50] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:51:05.275659 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:05.290747 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018667 Oct 16 00:51:05.291385 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 26/51] 10.222.0.25 () {60 vars in 1210 bytes} [Sun Oct 16 00:51:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:51:05.300929 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-782c964c-c4f2-4fea-8b6d-eeb1570ccee0 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:05.306558 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.wsgi [None req-782c964c-c4f2-4fea-8b6d-eeb1570ccee0 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-782c964c-c4f2-4fea-8b6d-eeb1570ccee0 admin admin] Request method failure captured: Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept: application/json Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Accept-Encoding: gzip, deflate Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Connection: keep-alive Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Content-Length: 0 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Host: 10.222.0.25 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-Api-Version: compute 2.92 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Openstack-System-Scope: None Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: User-Agent: python-novaclient Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Auth-Token: gAAAAABjS1V4XGbgD1j15pBFZIMFb0MUmHUUjY2lMJK5FbO29AInMdf_RrnfFsoEiFUdvaCIFFCjW5xI673jVyyLqOsmzkOy97iu2dGB-nSWH_NVU92Uc0JSBeQKU-Hau5PY8E9_AzkpuPpJaDPrk7pVAgwZhtoBx7-gd5hXTL30LDdpYzU3WWQ Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Id: None Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Domain-Name: None Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Identity-Status: Confirmed Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Is-Admin-Project: True Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Openstack-Nova-Api-Version: 2.92 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Id: default Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Domain-Name: Default Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Project-Name: admin Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Role: admin,reader,member Oct 16 00:51:05.308110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Roles: admin,reader,member Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/f1d89d5c90cd4c35b26fbcfc82450844"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_f1d89d5c90cd4c35b26fbcfc82450844", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/f1d89d5c90cd4c35b26fbcfc82450844"}]}] Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant: admin Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Id: f1d89d5c90cd4c35b26fbcfc82450844 Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-Tenant-Name: admin Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User: admin Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Id: default Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Domain-Name: Default Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Id: 68c5eeb505a240daa471059d6635deea Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: X-User-Name: admin Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: method: > Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: exception: Flavor 451 could not be found. Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: action_args: {'id': '451'} Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:51:05.309690 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:05.310777 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: ERROR nova.api.openstack.wsgi Oct 16 00:51:05.312221 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-782c964c-c4f2-4fea-8b6d-eeb1570ccee0 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=103763) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:51:05.313459 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-782c964c-c4f2-4fea-8b6d-eeb1570ccee0 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012168 Oct 16 00:51:05.313459 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 26/52] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:51:05 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 16 00:51:06.565564 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:06.566578 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Oct 16 00:51:06.567017 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 27/53] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:07.107740 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-12328f20-6cdc-4f2f-8286-34dc3e38d430 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:51:07.127475 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-12328f20-6cdc-4f2f-8286-34dc3e38d430 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.420734 Oct 16 00:51:07.128032 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 27/54] 10.222.0.25 () {64 vars in 1235 bytes} [Sun Oct 16 00:51:06 2022] POST /compute/v2.1/flavors => generated 404 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:51:08.375904 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:08.376943 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Oct 16 00:51:08.377355 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 28/55] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:08.912268 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-23fe4890-d4fe-45ef-907e-40afa98e540b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:51:08.931628 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-23fe4890-d4fe-45ef-907e-40afa98e540b admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.416482 Oct 16 00:51:08.932204 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 28/56] 10.222.0.25 () {64 vars in 1235 bytes} [Sun Oct 16 00:51:08 2022] POST /compute/v2.1/flavors => generated 405 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:51:10.185965 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:10.187092 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Oct 16 00:51:10.187598 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 29/57] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:10 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:10.723185 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-3dcc0cb1-8957-489f-9ab7-32c934046004 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:51:10.743591 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-3dcc0cb1-8957-489f-9ab7-32c934046004 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.417491 Oct 16 00:51:10.744148 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 29/58] 10.222.0.25 () {64 vars in 1235 bytes} [Sun Oct 16 00:51:10 2022] POST /compute/v2.1/flavors => generated 407 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:51:12.008292 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:12.009301 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001476 Oct 16 00:51:12.009700 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 30/59] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:12.552514 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-7117f9c3-7494-4056-b41d-9fd379410887 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:51:12.571808 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-7117f9c3-7494-4056-b41d-9fd379410887 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.421735 Oct 16 00:51:12.572211 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 30/60] 10.222.0.25 () {64 vars in 1235 bytes} [Sun Oct 16 00:51:12 2022] POST /compute/v2.1/flavors => generated 409 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:51:13.827657 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:13.828652 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Oct 16 00:51:13.829048 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 31/61] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:51:13 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:51:14.370330 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-b3a98c7c-9661-4121-a7ff-9fe00ca417e5 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:51:14.392126 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-b3a98c7c-9661-4121-a7ff-9fe00ca417e5 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.421789 Oct 16 00:51:14.392685 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 31/62] 10.222.0.25 () {60 vars in 1202 bytes} [Sun Oct 16 00:51:13 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:52:12.293061 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.294073 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-debc7f84-da6b-41b3-a63a-83ae313f4917 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001723 Oct 16 00:52:12.294569 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 32/63] 10.222.0.25 () {54 vars in 931 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:52:12.306809 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:52:12.723984 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-1c70a085-4177-4fc7-9bf5-569ee8d77478 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.745723 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-1c70a085-4177-4fc7-9bf5-569ee8d77478 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.443476 Oct 16 00:52:12.746276 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 32/64] 10.222.0.25 () {60 vars in 1292 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:52:12.767609 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_api:SELECT=14 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:52:12.771389 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-4f698360-1079-4ad2-9379-074662863bca admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.778834 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-4f698360-1079-4ad2-9379-074662863bca admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012635 Oct 16 00:52:12.779406 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 33/65] 10.222.0.25 () {60 vars in 1283 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:52:12.791348 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-53792ea4-8462-4fe0-941a-7f30043ddfcc admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.798729 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-53792ea4-8462-4fe0-941a-7f30043ddfcc admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010974 Oct 16 00:52:12.799292 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 33/66] 10.222.0.25 () {60 vars in 1286 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:52:12.811938 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.820349 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012012 Oct 16 00:52:12.820828 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 34/67] 10.222.0.25 () {60 vars in 1286 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:52:12.832939 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-187ba893-d811-4d81-a4bf-c0ac96cbb06f admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:52:12.840291 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-187ba893-d811-4d81-a4bf-c0ac96cbb06f admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011111 Oct 16 00:52:12.840827 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 34/68] 10.222.0.25 () {60 vars in 1283 bytes} [Sun Oct 16 00:52:12 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:55:11.782692 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:11.783725 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Oct 16 00:55:11.784152 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 35/69] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:55:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:55:11.924852 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=6 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:55:12.085467 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:12.090074 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:12.090535 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:12.090993 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:12.091693 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:12.092134 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:12.092544 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:12.124473 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-d9cf2ffd-ade8-498c-bafd-6348b702fcbb admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.200821 Oct 16 00:55:12.125024 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 35/70] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:55:11 2022] GET /compute/v2.1/os-services => generated 791 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:55:13.229869 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:13.230895 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Oct 16 00:55:13.231332 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 36/71] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:55:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:55:13.424928 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:13.429176 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:13.429501 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:13.429930 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:13.430669 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:13.431134 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:13.431549 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:13.458749 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-a665da2c-4505-4de6-9e75-ab76fe8a0ff8 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.085560 Oct 16 00:55:13.459281 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 36/72] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:55:13 2022] GET /compute/v2.1/os-services => generated 791 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:55:24.959530 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:24.960590 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001479 Oct 16 00:55:24.961201 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 37/73] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:55:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:55:25.105682 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_cell0:SELECT=4 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:55:25.106397 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_cell1:SELECT=4 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:55:25.107110 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG dbcounter [-] [103763] Writing DB stats nova_api:SELECT=8 {{(pid=103763) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:55:25.156701 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:25.160078 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:25.160549 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:25.160972 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:25.161863 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:25.162114 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:25.162526 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:25.187493 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-21413d4e-5476-421c-aa54-aac9c96fd1b7 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.086099 Oct 16 00:55:25.187977 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 37/74] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:55:25 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:55:26.288759 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG nova.api.openstack.wsgi [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] Calling method '>' {{(pid=103764) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:26.289757 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: INFO nova.api.openstack.requestlog [None req-b6d7be48-41b5-41a3-aba4-4dd37e308f74 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001415 Oct 16 00:55:26.290149 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: [pid: 103764|app: 0|req: 38/75] 10.222.0.25 () {56 vars in 1074 bytes} [Sun Oct 16 00:55:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:55:26.484941 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG nova.api.openstack.wsgi [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Calling method '>' {{(pid=103763) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:55:26.489297 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:26.489791 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:26.490232 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:26.490971 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:55:26.491421 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:55:26.491838 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: DEBUG oslo_concurrency.lockutils [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=103763) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:55:26.513406 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: INFO nova.api.openstack.requestlog [None req-26ba1dbc-119e-4af6-bb66-e13aa1b80bbc admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.080234 Oct 16 00:55:26.513886 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: [pid: 103763|app: 0|req: 38/76] 10.222.0.25 () {60 vars in 1193 bytes} [Sun Oct 16 00:55:26 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:56:26.473519 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: Sun Oct 16 00:56:26 2022 - graceful shutdown triggered... Oct 16 00:56:26.473663 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 16 00:56:26.474475 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: Gracefully killing worker 2 (pid: 103764)... Oct 16 00:56:26.474852 dv-cinder-master-3-iscsi devstack@n-api.service[103763]: Gracefully killing worker 1 (pid: 103763)... Oct 16 00:56:26.602968 dv-cinder-master-3-iscsi devstack@n-api.service[103764]: DEBUG dbcounter [-] [103764] Writing DB stats nova_api:SELECT=4 {{(pid=103764) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:56:27.474987 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: worker 1 buried after 1 seconds Oct 16 00:56:27.475496 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: worker 2 buried after 1 seconds Oct 16 00:56:27.475496 dv-cinder-master-3-iscsi devstack@n-api.service[103762]: goodbye to uWSGI. Oct 16 00:56:27.476557 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 16 00:56:27.477148 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 16 00:56:27.479962 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 16 00:56:27.489181 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 16 00:56:27 2022] *** Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: nodename: dv-cinder-master-3-iscsi Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: machine: x86_64 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: clock source: unix Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: pcre jit disabled Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: detected number of CPU cores: 4 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: current working directory: / Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: detected binary path: /usr/bin/uwsgi-core Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: your processes number limit is 47680 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: your memory page size is 4096 bytes Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: detected max file descriptor number: 2048 Oct 16 00:56:27.491922 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: lock engine: pthread robust mutexes Oct 16 00:56:27.493033 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: thunder lock: enabled Oct 16 00:56:27.493033 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 16 00:56:27.493033 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 16 00:56:27.532959 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: Python main interpreter initialized at 0x55852b3039c0 Oct 16 00:56:27.532959 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: python threads support enabled Oct 16 00:56:27.532959 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: your server socket listen backlog is limited to 100 connections Oct 16 00:56:27.532959 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: your mercy for graceful operations on workers is 90 seconds Oct 16 00:56:27.533414 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: mapped 403077 bytes (393 KB) for 2 cores Oct 16 00:56:27.533414 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: *** Operational MODE: preforking *** Oct 16 00:56:27.536679 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: *** uWSGI is running in multiple interpreter mode *** Oct 16 00:56:27.536679 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: spawned uWSGI master process (pid: 118995) Oct 16 00:56:27.536679 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: spawned uWSGI worker 1 (pid: 119012, cores: 1) Oct 16 00:56:27.536679 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: spawned uWSGI worker 2 (pid: 119013, cores: 1) Oct 16 00:56:27.536679 dv-cinder-master-3-iscsi devstack@n-api.service[118995]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 16 00:56:27.533918 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 16 00:56:28.218233 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 16 00:56:28.225622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 16 00:56:29.401928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:56:29.401928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: from cryptography.utils import int_from_bytes Oct 16 00:56:29.402522 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:56:29.402522 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: from cryptography.utils import int_from_bytes Oct 16 00:56:29.427672 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:56:29.427672 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: from cryptography.utils import int_from_bytes Oct 16 00:56:29.428265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 16 00:56:29.428265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: from cryptography.utils import int_from_bytes Oct 16 00:56:31.891928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 16 00:56:31.892529 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 16 00:56:31.892751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 16 00:56:31.893130 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 16 00:56:31.893671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 16 00:56:31.894335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.894938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.895513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.896060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.896577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.897087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.897602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.897940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.898280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.898728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.899090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.899608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.899951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.900279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.900607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.900942 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.901263 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.901602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.901936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.902459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.902956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.903279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.903604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.904227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.904346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.904584 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.904914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.905225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.905606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.905931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.907028 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.907457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.907827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.908145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.908597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.908916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.909241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.909571 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.909907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.910241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.910568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.910979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.911262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.911592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.911903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.912221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.912540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.912918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.913232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.913545 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.913809 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.914123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.914457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.914791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.915158 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.915418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.915677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.916006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.916335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.916653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.916968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.917281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.917602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.917913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.918224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.918539 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.918852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.919207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.919531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.919861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.920176 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.920501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.920921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.921178 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.921492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.921846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.922727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.923086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.923476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.923928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.924250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.924569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.924932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.925220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.925572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.925888 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.926145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.926459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.926774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.927132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.927447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.927768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.928081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.928390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.928712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.929019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.929332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.929645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.929957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.930276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.930585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.930974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.931320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.931657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.931995 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.932327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.932642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.932959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.933274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.933588 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.933919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.934234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.934548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.934897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.935221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.935539 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.935866 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.936181 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.936497 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.936841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.937133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.937488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.937870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.938340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.938770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.938983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.939285 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.939601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.939913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.940222 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.940535 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.940840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.941169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.941496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.941812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.942125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.942449 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.942762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.943138 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.943481 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.943839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.944159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.944504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.944844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.945167 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.945489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.945811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.946141 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.946459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.946780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.947427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.947427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.947767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.948110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.948428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.948754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.949081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.949398 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.949723 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.950061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.950392 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.950713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.951092 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.951497 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.951821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.952415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.952762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.953081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.953458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.953784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.954140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.954460 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.954780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.955133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.955457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.955806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.956114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.956435 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.956757 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.957079 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.957400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.957746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.958080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.958415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.958740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.959092 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.959420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.959743 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.960062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.960382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.960703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.961016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.961330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.961651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.961967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.962791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.962791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.963121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.963270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 16 00:56:31.963756 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 16 00:56:31.963913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.963913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.964200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 16 00:56:31.964339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.964485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 16 00:56:31.964623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.964768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 16 00:56:31.964917 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.965086 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.965222 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.965359 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.965505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.965651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.965796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.965796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.966065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.966201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.966349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.966491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.966641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.966778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.966956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.967092 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.967242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.967383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.967613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.967613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.967876 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.968020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.968162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.968162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.968465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.968603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.968750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.969448 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.969587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.969863 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.970039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.970227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.970368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.970506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.970640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.970789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.970951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.971139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.971242 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.971587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.971678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.971909 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.972002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.972249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.972349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.972563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.972668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.972899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.972990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.973213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.973304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.973524 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.973631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.973836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.973938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.974151 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.974249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.974463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.974562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.974781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.974899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.975120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.975230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.975434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.975535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.975817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.975989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.976136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.976282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.976371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.976534 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.976670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.976831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.976990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.977148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.977284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.977475 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.977598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.977803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.977902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.978119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.978254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.978455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.978554 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.978782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.978932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.979131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.979281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.979458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.979609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.979764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.979944 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.980614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.980728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.980966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.981062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.981147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.981241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.981341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.981427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.981518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.981621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.981723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.981816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.981909 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.982105 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.982199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.982423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.982515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.982794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.983010 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.983113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.983207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.983314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.983447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.983678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.983815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.984029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.984164 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.984341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.984481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.985198 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.985355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.985567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.985701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.985912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.986020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.986277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.986390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.986558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.986649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.986915 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.987030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.987200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.987320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.987516 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.987610 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.987834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.987934 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.988156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.988255 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.988463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.988599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.988784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.988926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.989102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.989194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.989426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.989524 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.989736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.989841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.990065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.990170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.990383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.990481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.990693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.990791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.991041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.991146 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.991364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.991456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.991678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.991770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.991991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.992090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.992303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.992395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.992628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.992720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.992954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.993052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.993262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.993361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.993518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.993619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.993823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.993915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.994133 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.994237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.994460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.994559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.994772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.994920 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.995117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.995290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.995429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.995527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.995735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.995835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.996045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.996149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.996385 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.996501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.996660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.996760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.996965 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.997066 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.997272 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.997374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.997583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.997730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.997891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.997993 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.998200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.998302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.998510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.998602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.998820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.998933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.999170 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.999264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.999482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:31.999574 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.999792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:31.999926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.000104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.000325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.000470 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.000614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.000720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.000814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.001037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.001234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.001387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.001531 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.001856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.002014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.002187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.002335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.002506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.002650 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.002820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.003046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.003156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.003374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.003468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.003704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.003796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.004027 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.004134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.004349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.004450 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.004682 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.004782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.005004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.005111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.005321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.005412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.005636 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.005735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.005947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.006054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.006258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.006406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.006569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.006664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.006896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.007005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.007211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.007347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.007520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.007613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.007828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.007921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.008147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.008239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.008466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.008559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.008777 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.008903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.009089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.009371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.009474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.009739 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 16 00:56:32.009840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010396 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010765 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.010870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011215 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011785 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.011886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.012856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.013086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.013204 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.013363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.013465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.013686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014251 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014677 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.014916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.015027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.015234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.015327 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.015548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.015647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.016377 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.016493 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.016693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.016900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017015 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017342 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.017994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.018222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.018330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.018546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.018637 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.018869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019010 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.019954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.020918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.021890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022430 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022521 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022743 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.022849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.023812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024120 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024439 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024752 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.024973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.025915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.026910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027010 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027297 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.027988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.028925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029160 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029391 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.029797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030151 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.030935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.031041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.031264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.032861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033763 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.033984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.034956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.035939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.036910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.037951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038209 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.038808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039166 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.039803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040327 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040435 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040742 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.040943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.041980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.042968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043141 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043248 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043787 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.043896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044512 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.044826 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.045996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.046099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.046320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.046501 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.046661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.046770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047334 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047448 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.047968 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048127 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.048924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049422 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049743 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.049869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050514 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.050856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051694 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.051786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.052997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053503 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053718 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.053940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054564 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.054899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055224 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055547 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.055998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.056189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.056319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.056501 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.056641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.056846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057580 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.057759 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.058794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059644 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.059959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.060899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061319 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061522 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.061931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.062151 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.062244 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.062471 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.062573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.062972 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.063910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064236 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.064919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065129 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.065875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.066105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.066205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.066424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.066721 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.066926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067484 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.067804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.068997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069111 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.069959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070460 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.070993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071565 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071788 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.071894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072425 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.072847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073166 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.073875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074013 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.074982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.075265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.075375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.075612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.075714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.075941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.076042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.076262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.076362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.076594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.076697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.077611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.077739 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.077739 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.077911 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.078005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.078104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.078197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.078885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.078997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.079900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.080946 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081167 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.081898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.082829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083082 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.083835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.084956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085048 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085365 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085887 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.085979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.086996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087275 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087644 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.087934 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.088870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.089814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.090787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.091949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092354 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092580 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092682 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.092986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.093206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.093309 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.093517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.093619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.093840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.094582 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.094713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.094977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.095975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.096908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.097954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.098936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.099131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.099298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.099448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.099647 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.099762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100459 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.100996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101338 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.101959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102077 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.102985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103282 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103590 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103804 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.103912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104236 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.104864 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.105822 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.106786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.107915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.108933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109592 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.109912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.110155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.110295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.110551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.110685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.110893 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111000 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111231 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111561 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.111883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112192 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.112967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113611 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.113926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.114095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.114328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.114470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.114690 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115734 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.115966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116392 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116694 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.116922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117344 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.117960 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118176 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.118932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.119883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120196 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120734 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.120842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.121994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122617 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.122962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123285 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123707 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.123973 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124289 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124449 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.124901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.125064 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.125280 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.125418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.125616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.125918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126224 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126514 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.126962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127524 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.127964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128853 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.128960 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.129239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.129401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.129572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.129746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.129906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.130335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.130510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.130654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.130747 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.130836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131013 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131378 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.131859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.132991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133093 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.133949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.134925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135400 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.135883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136519 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.136994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137519 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137658 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.137949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.138174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.138304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.138522 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.138618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.138824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139895 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.139993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140173 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.140689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.141349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.141559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.141659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.141849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142163 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142293 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.142952 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143154 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.143902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144570 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.144889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.145864 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.146825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147068 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.147829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148029 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148462 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.148856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149005 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149158 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.149884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.150982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.151990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.152895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153330 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.153935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154392 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154773 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.154936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.155962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.156107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.156495 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.157186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.157342 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.157480 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.157623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.157770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.158000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.158136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.158316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.158662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.158808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159682 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.159975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.160939 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161571 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.161988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.162213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.162316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.162541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.162659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.162853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163270 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.163942 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164174 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164511 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.164941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.165151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.165247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.165473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.165575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.165791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.166835 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167750 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.167851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168160 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168579 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.168812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169339 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169654 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.169975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170079 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.170971 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119013) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 16 00:56:32.171087 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.171295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.171615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.171937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.172400 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.172940 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.173265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.173612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.173952 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.174285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.174613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.174962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.175301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.175625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.175942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.176264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.176582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.176896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.177227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.177548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.177858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.178185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.178573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.179070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.179427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.179755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.180088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.180420 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.180743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.181142 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.181388 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.181699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.182011 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.182335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.182708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.182976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.183291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.183611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.183928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.184306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.184554 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.184865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.185203 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.185519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.185856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.186168 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.186487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.186798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.187135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.187447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.187877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.188325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.188646 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.189067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.189385 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.189704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.190067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.190386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.190698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.191060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.191378 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.191690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.192021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.192340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.192671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.192992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.193307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.193665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.193989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.194564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.194937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.195278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.195620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.195959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.196274 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.196587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.196899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.197232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.197548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.197867 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.198185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.198501 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.198816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.199157 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.199474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.199788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.200104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.200419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.200797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.201086 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.201402 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.201715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.202301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.202653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.203000 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.203895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.204219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.204545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.204865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.205186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.205503 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.205821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.206143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.206479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.206799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.207136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.207578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.207901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.208230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.208557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.208879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.209207 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.209536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.209857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.210185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.210507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.210845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.211211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.212578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.212578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.212578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.212578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.213019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.213114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.213377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.213690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.214000 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.214312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.214629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.214966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.215281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.215591 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.215910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.216227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.216538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.216849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.217155 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.217464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.217773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.218078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.218395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.218703 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.219125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.219946 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.220343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.220655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.220998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.221339 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.221663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.221994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.222313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.222738 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.222982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.223332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.223648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.223978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.224298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.224641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.224967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.225284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.225603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.225923 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.226241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.226564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.226981 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.227318 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.227631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.227961 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.228295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.228613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.228936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.229252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.229589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.229907 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.230223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.230542 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.230880 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.231194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.231515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.231831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.232144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.232465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.232814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.233121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.233453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.233772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.234103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.234436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.234601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO dbcounter [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Registered counter for database nova_api Oct 16 00:56:32.235035 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.235369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.235728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.236071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.236396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.236717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.237058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 16 00:56:32.237328 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=119012) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 16 00:56:32.255566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_db.sqlalchemy.engines [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119013) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.262181 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writer thread running {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.303970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO dbcounter [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Registered counter for database nova_api Oct 16 00:56:32.323311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.context [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d19e9fc7-2682-4022-903d-afef81878fc4(cell1) {{(pid=119013) load_cells /opt/stack/nova/nova/context.py:464}} Oct 16 00:56:32.324753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_db.sqlalchemy.engines [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119012) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.328251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:56:32.329732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:56:32.331653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:56:32.332434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:56:32.332910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:56:32.333225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writer thread running {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.335651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:56:32.337903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO dbcounter [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Registered counter for database nova_cell0 Oct 16 00:56:32.342347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO dbcounter [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Registered counter for database nova_cell1 Oct 16 00:56:32.351853 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_db.sqlalchemy.engines [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119013) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.352762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_db.sqlalchemy.engines [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119013) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.359133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writer thread running {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.394660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.context [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),d19e9fc7-2682-4022-903d-afef81878fc4(cell1) {{(pid=119012) load_cells /opt/stack/nova/nova/context.py:464}} Oct 16 00:56:32.398847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:56:32.399363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:56:32.400988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:56:32.401752 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:56:32.402227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:56:32.404323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:56:32.406342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO dbcounter [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Registered counter for database nova_cell0 Oct 16 00:56:32.413498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO dbcounter [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Registered counter for database nova_cell1 Oct 16 00:56:32.422618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_db.sqlalchemy.engines [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119012) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.425257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_db.sqlalchemy.engines [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119012) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.434449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writer thread running {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.535595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writer thread running {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.545931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO dbcounter [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Registered counter for database nova_cell0 Oct 16 00:56:32.554655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_db.sqlalchemy.engines [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119013) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.560067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writer thread running {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.595941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writer thread running {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.608281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO dbcounter [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Registered counter for database nova_cell0 Oct 16 00:56:32.622568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_db.sqlalchemy.engines [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=119012) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 16 00:56:32.630771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writer thread running {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 16 00:56:32.980609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: WARNING keystonemiddleware.auth_token [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:56:32.988971 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: WARNING keystonemiddleware.auth_token [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:56:33.027161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: WARNING keystonemiddleware.auth_token [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:56:33.035301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: WARNING keystonemiddleware.auth_token [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:56:33.357402 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: WARNING keystonemiddleware.auth_token [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:56:33.364418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: WARNING keystonemiddleware.auth_token [None req-b60f6e02-6d3c-4e80-9d94-593eb977fba3 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:56:33.369753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55852b3039c0 pid: 119012 (default app) Oct 16 00:56:33.387891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: WARNING keystonemiddleware.auth_token [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 16 00:56:33.394893 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: WARNING keystonemiddleware.auth_token [None req-3390c2ea-88d7-4d0a-807f-0317aea06d05 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 16 00:56:33.400297 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55852b3039c0 pid: 119013 (default app) Oct 16 00:57:12.439664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:12.442552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:12.444540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:12.445957 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:13.698254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:13.699871 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:13.704881 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:13.706198 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:14.665872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-57a3e9fa-697b-415a-81ce-c2dd6fc33b85 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Action: 'create', calling method: .version_select at 0x7f8bbea25550>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1385670240", "metadata": {"Type": "work"}}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-57a3e9fa-697b-415a-81ce-c2dd6fc33b85 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 122 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:14.676478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1385670240", "metadata": {"Type": "work"}}} Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:14.678025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1385670240', 'metadata': {'Type': 'work'}}}} Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise error Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi '#$%' Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:14.678949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 16 00:57:14.680347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 16 00:57:14.681543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 16 00:57:14.681543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.681543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-57a3e9fa-697b-415a-81ce-c2dd6fc33b85 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:14.681543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-57a3e9fa-697b-415a-81ce-c2dd6fc33b85 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 2.267627 Oct 16 00:57:14.681543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 1/1] 10.222.0.25 () {60 vars in 1156 bytes} [Sun Oct 16 00:57:12 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 2271 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 16 00:57:14.711042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-77a82b5d-acb9-4241-9e54-fcddb0d4a1d1 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Action: 'create', calling method: .version_select at 0x7f8bbea25e50>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-87221875", "metadata": {"Type": "work"}}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-77a82b5d-acb9-4241-9e54-fcddb0d4a1d1 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 118 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:14.716020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-87221875", "metadata": {"Type": "work"}}} Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:14.719281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-87221875', 'metadata': {'Type': 'work'}}}} Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise error Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi '0' Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:14.720770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 16 00:57:14.723438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 16 00:57:14.726268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 16 00:57:14.726268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.726268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-77a82b5d-acb9-4241-9e54-fcddb0d4a1d1 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:14.726268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-77a82b5d-acb9-4241-9e54-fcddb0d4a1d1 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.031565 Oct 16 00:57:14.726268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 2/2] 10.222.0.25 () {60 vars in 1156 bytes} [Sun Oct 16 00:57:14 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 38 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 16 00:57:14.752776 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2c7ff50d-f341-4ba6-adc0-613c9268bff5 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Action: 'create', calling method: .version_select at 0x7f8bbe8e1280>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2044228094", "metadata": {"Type": "work"}}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2c7ff50d-f341-4ba6-adc0-613c9268bff5 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 119 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:14.762947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-2044228094", "metadata": {"Type": "work"}}} Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:14.766119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-2044228094', 'metadata': {'Type': 'work'}}}} Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise error Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi '' Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:14.767440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 16 00:57:14.770371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 16 00:57:14.774206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 16 00:57:14.774206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:14.774206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2c7ff50d-f341-4ba6-adc0-613c9268bff5 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:14.774206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2c7ff50d-f341-4ba6-adc0-613c9268bff5 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.042531 Oct 16 00:57:14.774206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 3/3] 10.222.0.25 () {60 vars in 1156 bytes} [Sun Oct 16 00:57:14 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 47 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 16 00:57:14.795940 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-37642e59-6edf-4df8-80d5-b7dfd51dd4b0 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Calling method '.version_select at 0x7f8bbe8e1ca0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:15.217805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-480ea4fd-be77-48f6-8912-5f8b94eba538 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Action: 'create', calling method: .version_select at 0x7f8bbea22550>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1372742649", "metadata": {"Type": "work"}}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:15.539019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-37642e59-6edf-4df8-80d5-b7dfd51dd4b0 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] HTTP exception thrown: Volume tempest-invalid-1693700706 could not be found. Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-37642e59-6edf-4df8-80d5-b7dfd51dd4b0 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1693700706 HTTP/1.1 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:15.544502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Volume tempest-invalid-1693700706 could not be found. Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'tempest-invalid-1693700706'} Oct 16 00:57:15.546465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1693700706 could not be found. (HTTP 404) (Request-ID: req-ee8254d0-8360-41ce-97f0-dd2f376cf919) Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:15.547878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 00:57:15.550323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1693700706 could not be found. Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:57:15.553726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1693700706 could not be found. Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1693700706 could not be found. Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-37642e59-6edf-4df8-80d5-b7dfd51dd4b0 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 404 to user: Volume tempest-invalid-1693700706 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-37642e59-6edf-4df8-80d5-b7dfd51dd4b0 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1693700706" status: 404 len: 99 microversion: 2.1 time: 0.768140 Oct 16 00:57:15.557516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 4/4] 10.222.0.25 () {58 vars in 1218 bytes} [Sun Oct 16 00:57:14 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1693700706 => generated 99 bytes in 771 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Oct 16 00:57:15.564210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8cd056c4-3092-4f5b-8dce-934db07b900e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.008307 Oct 16 00:57:15.566004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 5/5] 10.222.0.25 () {58 vars in 1140 bytes} [Sun Oct 16 00:57:15 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 11 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 16 00:57:15.587435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6430a1eb-68df-4bd4-b707-3db59f849a3c tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.009122 Oct 16 00:57:15.594798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 6/6] 10.222.0.25 () {58 vars in 1137 bytes} [Sun Oct 16 00:57:15 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 20 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 16 00:57:15.611633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e8c71b55-e575-4faa-828c-4be689f5ca25 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Calling method '.version_select at 0x7f8bbe865ee0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:16.523261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-e8c71b55-e575-4faa-828c-4be689f5ca25 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] HTTP exception thrown: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e8c71b55-e575-4faa-828c-4be689f5ca25 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/os-volumes/68f299e8-1341-45f9-b042-f129b6b6a2dd HTTP/1.1 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:16.532135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '68f299e8-1341-45f9-b042-f129b6b6a2dd'} Oct 16 00:57:16.533595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. (HTTP 404) (Request-ID: req-c8d59133-d7df-4425-ae3e-108df12094ba) Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.534404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 00:57:16.535573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 16 00:57:16.536691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e8c71b55-e575-4faa-828c-4be689f5ca25 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 404 to user: Volume 68f299e8-1341-45f9-b042-f129b6b6a2dd could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:16.540268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e8c71b55-e575-4faa-828c-4be689f5ca25 tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/68f299e8-1341-45f9-b042-f129b6b6a2dd" status: 404 len: 109 microversion: 2.1 time: 0.937901 Oct 16 00:57:16.549607 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 7/7] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 00:57:15 2022] DELETE /compute/v2.1/os-volumes/68f299e8-1341-45f9-b042-f129b6b6a2dd => generated 109 bytes in 954 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:57:16.570353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2cf68492-3713-4487-9c74-dfd926f1d87e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Calling method '.version_select at 0x7f8bbe816430>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:16.731304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-2cf68492-3713-4487-9c74-dfd926f1d87e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] HTTP exception thrown: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2cf68492-3713-4487-9c74-dfd926f1d87e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Request method failure captured: Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/os-volumes/6b98c357-6129-4759-8551-c122a674c706 HTTP/1.1 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1box5LXq49bCqj-f7ZPk-7qy3SFNYfWZaDekaM9BvcGzK34OWPTLTVyujUlqJqXrJ1aOKPFygEkK6Wj-Cq10qed_OG9HiYhLJKoQJv72taRUbpEkAHOakHt8itxKaFKKsJXCp1eWlQnspbfwKLWxXgUyvLimAfk6WEccQsYX0YWc00 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:57:16.737745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/0d378a01be2044faa872ca43f816dbd9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_0d378a01be2044faa872ca43f816dbd9", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/0d378a01be2044faa872ca43f816dbd9"}]}] Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 0d378a01be2044faa872ca43f816dbd9 Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-260776570 Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ca7148520e6b4c3e8f5400a2d79ebc87 Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-260776570-project Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '6b98c357-6129-4759-8551-c122a674c706'} Oct 16 00:57:16.740664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. (HTTP 404) (Request-ID: req-b23639c3-0f59-44d0-8342-6e4d484f225a) Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.742453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 00:57:16.745303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 16 00:57:16.751062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:57:16.754988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2cf68492-3713-4487-9c74-dfd926f1d87e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] Returning 404 to user: Volume 6b98c357-6129-4759-8551-c122a674c706 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:16.762728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2cf68492-3713-4487-9c74-dfd926f1d87e tempest-VolumesNegativeTest-260776570 tempest-VolumesNegativeTest-260776570-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/6b98c357-6129-4759-8551-c122a674c706" status: 404 len: 109 microversion: 2.1 time: 0.210172 Oct 16 00:57:16.764782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 8/8] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:16 2022] GET /compute/v2.1/os-volumes/6b98c357-6129-4759-8551-c122a674c706 => generated 109 bytes in 215 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:57:17.486660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-480ea4fd-be77-48f6-8912-5f8b94eba538 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 3.803802 Oct 16 00:57:17.504165 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 1/9] 10.222.0.25 () {60 vars in 1156 bytes} [Sun Oct 16 00:57:13 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 3824 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 16 00:57:17.516805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-613cd4f2-6b4a-4cd5-a809-6bbfba939cdb tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe8e4040>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:17.766349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-613cd4f2-6b4a-4cd5-a809-6bbfba939cdb tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 200 len: 353 microversion: 2.1 time: 0.254326 Oct 16 00:57:17.767559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 9/10] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:17 2022] GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 353 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:18.801645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-be5f8c33-e957-4e92-a07a-36b93a32bea1 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe8f8430>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:18.962549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-be5f8c33-e957-4e92-a07a-36b93a32bea1 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 200 len: 354 microversion: 2.1 time: 0.171638 Oct 16 00:57:18.963706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 2/11] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:18 2022] GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 354 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:18.977052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a7e5b08a-4194-45c2-9ccf-a4ac70252b17 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe8168b0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:19.230889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a7e5b08a-4194-45c2-9ccf-a4ac70252b17 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 200 len: 354 microversion: 2.1 time: 0.257676 Oct 16 00:57:19.231685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 10/12] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:18 2022] GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 354 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:19.251759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e8c101f2-8eaa-4b25-8f42-7538c0061562 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe8f8af0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:19.635594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e8c101f2-8eaa-4b25-8f42-7538c0061562 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 202 len: 0 microversion: 2.1 time: 0.387602 Oct 16 00:57:19.636266 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 3/13] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 00:57:19 2022] DELETE /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 0 bytes in 389 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 00:57:19.645456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1fce98ed-445d-4e2e-be8b-99a739a902d7 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe816040>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:19.863145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1fce98ed-445d-4e2e-be8b-99a739a902d7 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 200 len: 353 microversion: 2.1 time: 0.221628 Oct 16 00:57:19.864490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 11/14] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:19 2022] GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 353 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:20.881146 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0c80479f-d5dc-4e61-bd14-e328d10991e5 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Calling method '.version_select at 0x7f8bbe8911f0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:20.979433 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-0c80479f-d5dc-4e61-bd14-e328d10991e5 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] HTTP exception thrown: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0c80479f-d5dc-4e61-bd14-e328d10991e5 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Request method failure captured: Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 HTTP/1.1 Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1bpyQCyvASQDGzRQxoNyQbSgwkCo1lGKd6mttefFnPq7WiUFrwXp1yALa0d9Ssc745iuuxDVMQ3vi9TwkQVsu3n-WYehemtQZpRw5nJ0EJJLSFF9nsViU8qFyIhABuW-0TNcmD-18Vc916wFabIwEKudSiWRZGsnjdVLTDjrEnInmg Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: db213870f6a145cf9405415bdeb0a92b Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-VolumesGetTestJSON-657819979 Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 00:57:20.983963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/db213870f6a145cf9405415bdeb0a92b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/db213870f6a145cf9405415bdeb0a92b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_db213870f6a145cf9405415bdeb0a92b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/db213870f6a145cf9405415bdeb0a92b"}]}] Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-VolumesGetTestJSON-657819979 Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: db213870f6a145cf9405415bdeb0a92b Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-VolumesGetTestJSON-657819979 Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-VolumesGetTestJSON-657819979-project Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 63ebc9b0925a4b078f35f7faf3c5fd84 Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-VolumesGetTestJSON-657819979-project Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '5201fde0-266d-4961-b50e-e88ffd02d990'} Oct 16 00:57:20.985347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. (HTTP 404) (Request-ID: req-4f998341-0e69-4948-b855-bb574e4689d9) Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:20.986317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 00:57:20.987738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 16 00:57:20.988852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. Oct 16 00:57:20.990030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:57:20.992671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0c80479f-d5dc-4e61-bd14-e328d10991e5 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] Returning 404 to user: Volume 5201fde0-266d-4961-b50e-e88ffd02d990 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:57:20.993649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0c80479f-d5dc-4e61-bd14-e328d10991e5 tempest-VolumesGetTestJSON-657819979 tempest-VolumesGetTestJSON-657819979-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990" status: 404 len: 109 microversion: 2.1 time: 0.119368 Oct 16 00:57:20.995341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 4/15] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 00:57:20 2022] GET /compute/v2.1/os-volumes/5201fde0-266d-4961-b50e-e88ffd02d990 => generated 109 bytes in 122 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:57:25.872447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1932933974", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "networks": [{"uuid": "987da718-4a64-4801-99fd-c8b997ad845b"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:26.283580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] validate_networks() for [('987da718-4a64-4801-99fd-c8b997ad845b', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:57:30.445310 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:57:30.446078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:57:30.446283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:57:30.446602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:57:30.447187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:57:30.501726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:30.503231 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:30.503231 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:30.503497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:30.503897 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:30.506329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:30.623672 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Getting quotas for project c89fd0e945344feab91a37fdb93812e5. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:30.646790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Getting quotas for user 3aa7409c9f6f46f78164818b6f6927e4 and project c89fd0e945344feab91a37fdb93812e5. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:30.673144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:57:30.695198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:57:30.865089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a0480892-a3e0-42a3-9c3b-77409656b6af tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.534472 Oct 16 00:57:30.883904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ac92285d-c272-4ef4-8118-cea18d6713bd tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:30.884842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ac92285d-c272-4ef4-8118-cea18d6713bd tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:30.890035 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 12/16] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:57:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 5559 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:57:31.199860 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ac92285d-c272-4ef4-8118-cea18d6713bd tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:31.240136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ac92285d-c272-4ef4-8118-cea18d6713bd tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1215 microversion: 2.1 time: 0.359073 Oct 16 00:57:31.262503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 5/17] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:30 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1215 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:32.271249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-775ae9e1-4d6b-4bdd-a86f-adb6f57f4e18 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:32.273834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-775ae9e1-4d6b-4bdd-a86f-adb6f57f4e18 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:32.590190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-775ae9e1-4d6b-4bdd-a86f-adb6f57f4e18 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:32.624253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-775ae9e1-4d6b-4bdd-a86f-adb6f57f4e18 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1215 microversion: 2.1 time: 0.357617 Oct 16 00:57:32.639375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 13/18] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:32 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1215 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:33.665824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:33.666447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:33.680960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:33.683871 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:33.684798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:33.687040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:33.688648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:33.691673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:34.167906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:34.191879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-293b4db6-18f7-4c21-aaaf-30ffab8b5a10 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1219 microversion: 2.1 time: 0.530003 Oct 16 00:57:34.211257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 6/19] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:33 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1219 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:35.220584 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:35.222613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:35.269461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:35.273132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:35.274610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:35.280328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:35.280837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:35.283848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:35.478517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1841455897", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:35.617178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] validate_networks() for [('f6d9aa56-b330-4b62-a416-c067206c96f0', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:57:35.665674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:35.695565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-df7f7668-9645-46fe-953d-c0eea72867a3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1211 microversion: 2.1 time: 0.480058 Oct 16 00:57:35.699675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 7/20] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:35 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1211 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:36.568835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0ae5cf3e-454a-493e-9a7a-04aeea527f1d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: .version_select at 0x7f8bbe7ba4c0>, body: {"keypair": {"name": "tempest-keypair-1791417281", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHwCojS3GHju6Q62swLEuAVO0sJWEizrjy2qsv5YHgFPC1K+9z+tyvC0Qr5HSri48F/ssmiApru2OOJhYk4o5dw+snycUiAgRDOTrLaxiqfKAScKht5GFINM9D6kO1qosg=="}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:36.772921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-0ae5cf3e-454a-493e-9a7a-04aeea527f1d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:36.787933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-0ae5cf3e-454a-493e-9a7a-04aeea527f1d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:36.994549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0ae5cf3e-454a-493e-9a7a-04aeea527f1d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.132458 Oct 16 00:57:37.005875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 8/21] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 00:57:35 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:37.013243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:37.022759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:37.053605 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:37.054342 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:37.054616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:37.055903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:37.056379 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:37.064595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:37.618097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:37.633046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6056aef3-65bc-4b12-aa69-82728435e69b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1273 microversion: 2.1 time: 0.634151 Oct 16 00:57:37.639889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 9/22] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:36 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1273 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:37.756022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:57:37.757218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:57:37.757840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:57:37.758419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:57:37.759195 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:57:37.799217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:37.801731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:37.803908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:37.806056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:37.808252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:37.809589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:37.868803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:37.885065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for user b01ce2d27bdb4c91a3f55c891f949e00 and project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:37.947073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:57:37.950258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:57:38.048570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d61795e1-4e1e-41d2-a799-14e2fc1f16f5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.934113 Oct 16 00:57:38.049982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 14/23] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:57:35 2022] POST /compute/v2.1/servers => generated 383 bytes in 2936 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:57:38.064244 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c5fd26da-c189-4bc2-bc07-090930d4d760 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:38.067763 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c5fd26da-c189-4bc2-bc07-090930d4d760 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:38.741121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c5fd26da-c189-4bc2-bc07-090930d4d760 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:38.749360 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c5fd26da-c189-4bc2-bc07-090930d4d760 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1222 microversion: 2.1 time: 0.688879 Oct 16 00:57:38.751567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 10/24] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:38 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1222 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:38.773708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:38.774596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:38.808837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:38.809762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:38.811191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:38.812166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:38.812801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:38.813401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:39.315696 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:39.329235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c079f7f2-d173-46a5-bc87-3ab64c90a1e6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1273 microversion: 2.1 time: 0.573678 Oct 16 00:57:39.336711 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 11/25] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:38 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1273 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:39.384253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1914295994", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:39.492180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b47c92ef-af0a-4ee6-8ecc-6b3e1cb32bb4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: .version_select at 0x7f8bbdf20a60>, body: {"keypair": {"name": "tempest-keypair-8122179", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGESbsa1euMvo0sNlBWVoVpPlxCJvryOL6k1ljK/LlZa/Gsv0YwGYSCT87hqYQmCFKxpAxCjWN8NSZwDINK+nL2yZWpZFl3zz8wWI2PJKfWiUABIN/2lUhnfmoyINNdWWA=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:39.591142 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b47c92ef-af0a-4ee6-8ecc-6b3e1cb32bb4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:39.611647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b47c92ef-af0a-4ee6-8ecc-6b3e1cb32bb4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for user ccece85a85fb48c4887b52c96c5bc642 and project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:39.795059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] validate_networks() for [('f6d9aa56-b330-4b62-a416-c067206c96f0', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:57:39.806305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b47c92ef-af0a-4ee6-8ecc-6b3e1cb32bb4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 398 microversion: 2.20 time: 1.466025 Oct 16 00:57:39.810009 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 15/26] 10.222.0.25 () {62 vars in 1200 bytes} [Sun Oct 16 00:57:38 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 1473 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Oct 16 00:57:40.371953 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:40.373234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:40.384697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:40.388277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:40.390305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:40.392645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:40.394544 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:40.396414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:41.079650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:41.147324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e6f0d39a-b7e0-4c1d-8852-6c84e26c6de6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1273 microversion: 2.1 time: 0.773108 Oct 16 00:57:41.154787 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 16/27] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:40 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1273 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:42.175676 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:42.176460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:42.191421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:42.196298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:42.199846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:42.203578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:42.207150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:42.207634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:43.629109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:57:43.630184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:57:43.630935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:57:43.631645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:57:43.632357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:57:43.690821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:43.692585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:43.693053 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:43.693874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:43.694347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:43.707164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:43.792098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:43.802659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for user b01ce2d27bdb4c91a3f55c891f949e00 and project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:43.836443 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:57:43.845984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:57:43.993491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a9b2c772-3ca4-4304-bb11-c613b483bee7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.654940 Oct 16 00:57:43.998949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 12/28] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:57:39 2022] POST /compute/v2.1/servers => generated 383 bytes in 4661 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:57:44.007159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-86f0eba4-8596-49b6-9119-4325714f4485 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:44.009207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-86f0eba4-8596-49b6-9119-4325714f4485 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:44.182632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-86f0eba4-8596-49b6-9119-4325714f4485 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:44.195417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-86f0eba4-8596-49b6-9119-4325714f4485 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1222 microversion: 2.1 time: 0.192339 Oct 16 00:57:44.207715 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 13/29] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:44 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1222 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:44.780930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:44.814820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-74d79cde-e7f3-408b-8c82-181746d62e91 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 2.641140 Oct 16 00:57:44.818044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 17/30] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:42 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 2645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:45.246953 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-05caeab8-541b-41c4-9b1c-5c699be6fbed tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:45.251184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-05caeab8-541b-41c4-9b1c-5c699be6fbed tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:45.846998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:45.860126 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:45.875895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:45.878772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:45.879682 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:45.881316 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:45.884692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:45.885366 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:46.072803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-05caeab8-541b-41c4-9b1c-5c699be6fbed tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:46.094784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-05caeab8-541b-41c4-9b1c-5c699be6fbed tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1222 microversion: 2.1 time: 0.857333 Oct 16 00:57:46.095918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 14/31] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:45 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1222 bytes in 860 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:46.156875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-369291287", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7dae3d76-90a2-41d2-9dad-7f7856d5ab85"}], "security_groups": [{"name": "tempest-securitygroup--1921163935"}], "key_name": "tempest-keypair-1791417281", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:46.390640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:46.411013 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-27c9e46a-2b50-4ff7-8426-5ce38099de52 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.569602 Oct 16 00:57:46.418926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 18/32] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:45 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:47.125568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:47.128466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:47.145229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:47.152178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:47.154769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:47.159004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:47.162510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:47.165510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:47.358338 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:47.378523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-97ca63c3-0e86-489e-81f5-321da96af4c9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1274 microversion: 2.1 time: 0.256748 Oct 16 00:57:47.380891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 19/33] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:47 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1274 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:47.452649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:47.453662 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:47.479253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] validate_networks() for [('7dae3d76-90a2-41d2-9dad-7f7856d5ab85', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:57:47.487399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:47.489014 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:47.489489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:47.490252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:47.490720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:47.497838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:47.835332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:48.538647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:48.559080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3aec4064-1c53-4ce8-b3ab-64ecfdf63980 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 1.115151 Oct 16 00:57:48.559948 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 20/34] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:47 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 1117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:48.597606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:48.598247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:48.639845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:48.640354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:48.640805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:48.641531 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:48.641994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:48.642470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:48.867838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:48.885890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c64d9f13-6bbf-44ad-b34f-8fa843346530 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1280 microversion: 2.1 time: 0.305825 Oct 16 00:57:48.908701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 21/35] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:48 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1280 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:49.612369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:49.615245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:49.633726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:49.638599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:49.639358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:49.641581 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:49.642372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:49.643205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:49.660567 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=18 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:50.712225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:50.736114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-eaaf2c6a-c08f-45ea-9aa5-b0ba2cc3f2e7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 1.145561 Oct 16 00:57:50.740896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 22/36] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:49 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 1152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:50.755293 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:50.767976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:50.805383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:50.805895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:50.814852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:50.815689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:50.816189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:50.816631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:50.873420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:57:50.874016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:57:50.874502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:57:50.875007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:57:50.875487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:57:50.901072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:50.908016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:50.908626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:50.915837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:50.918783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:50.919402 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:50.997149 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:51.030708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:51.115110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:57:51.117710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:57:51.258315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:51.260937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-245ca12f-82a2-43d4-a5fe-d3d4301f39b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 5.136346 Oct 16 00:57:51.265541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 15/37] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:57:46 2022] POST /compute/v2.1/servers => generated 402 bytes in 5142 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 16 00:57:51.267524 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7abfc466-b4e7-42bc-a333-bf48590b7c4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1280 microversion: 2.1 time: 0.524165 Oct 16 00:57:51.271855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 23/38] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:50 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1280 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:51.295234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4800030d-185b-44b5-b630-4dd439a457f2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:51.307250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4800030d-185b-44b5-b630-4dd439a457f2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:51.552083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4800030d-185b-44b5-b630-4dd439a457f2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:51.560491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4800030d-185b-44b5-b630-4dd439a457f2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1241 microversion: 2.1 time: 0.276046 Oct 16 00:57:51.564849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 16/39] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:51 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1241 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:51.764948 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:51.766400 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:51.781145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:51.784955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:51.788372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:51.791603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:51.794431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:51.796563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:52.303661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:52.304396 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:52.332198 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:52.332779 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:52.333239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:52.334576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:52.339379 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:52.341659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:52.673586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:52.695563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dc06075b-46a6-450b-bdb5-65b038af2ee6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.936209 Oct 16 00:57:52.704446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 24/40] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:51 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:52.712686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:52.720491 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:52.738852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:52.740425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:52.741732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:52.742826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:52.744669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:52.746021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:53.627778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:53.637662 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9d1d8a43-fa4f-4402-b9de-ba2c87eb3c12 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1245 microversion: 2.1 time: 0.931626 Oct 16 00:57:53.640412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 25/41] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:52 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1245 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:53.735895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:53.741150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:53.773474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:53.774827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:53.776056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:53.778936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:53.780438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:53.782650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:54.540675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:54.554659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0928db8-20bb-4adc-b9d5-a6668e83498d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 2.259687 Oct 16 00:57:54.578088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 17/42] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:52 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 2273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:57:54.709040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:54.710386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:54.739297 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:54.745215 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:54.747534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:54.754037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:54.755997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:54.756667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:54.973390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:54.984269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d5665b0c-0ee2-4362-bafc-e7bb42cd859e tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 1.255907 Oct 16 00:57:54.997379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 26/43] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:53 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 1269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:55.014498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1967590964", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "758abdbd-543f-42fe-8aa7-53af6cbbeb5d"}], "security_groups": [{"name": "tempest-securitygroup--977466123"}], "key_name": "tempest-keypair-8122179", "user_data": "}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:55.152315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:55.161331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6952a958-a197-4eca-b25d-f55739b410b0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1299 microversion: 2.1 time: 0.457128 Oct 16 00:57:55.165366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 18/44] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:54 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1299 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:55.637323 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a9b3320-e119-44b4-8035-2726ec9b56d4", "tag": "67ec654d-ac07-4f60-a537-631325285925"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:55.691169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:55.693189 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:55.693922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:55.758840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] validate_networks() for [('758abdbd-543f-42fe-8aa7-53af6cbbeb5d', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:57:55.784904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Creating event network-changed:67ec654d-ac07-4f60-a537-631325285925 for instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 on hv-cinder-maste Oct 16 00:57:55.786998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Acquiring lock "compute-rpcapi-router" {{(pid=119013) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 16 00:57:55.788407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-712cf2f8-9173-437f-b5f7-ca13f1b7b1cc req-6c4f2ba2-86c2-4951-9095-f01659e3ddec service nova] Acquired lock "compute-rpcapi-router" {{(pid=119013) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 16 00:57:55.793146 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:55.799801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:55.799801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:55.799801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:55.799801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:55.800115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:55.825606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.compute.rpcapi [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 16 00:57:55.829255 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] Releasing lock "compute-rpcapi-router" {{(pid=119013) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 16 00:57:55.864616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e4106bd9-41b5-478e-869c-76f44ba6ed68 None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.509427 Oct 16 00:57:55.865221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 19/45] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:57:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:55.890247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:55.894082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:55.908610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:55.909119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:55.909566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:55.916550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:55.916550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:55.916550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:56.649974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=7 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:56.879408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:56.899544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6d03ea81-e7e5-471a-8c83-847aec269844 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 1.032780 Oct 16 00:57:56.903672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 20/46] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:55 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 1037 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:56.921745 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:56.922631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:56.941467 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:56.944988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:56.945544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:56.948627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:56.949060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:56.950100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:57.415183 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:57:57.415556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:57:57.416747 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:57:57.417179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:57:57.418055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:57:57.445733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:57.446232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:57.447440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:57.452954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:57.453444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:57.453882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:57.479687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=71 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:57:57.516601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:57:57.529923 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for user ccece85a85fb48c4887b52c96c5bc642 and project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:57:57.564800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:57:57.572426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:57:57.673374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f2c572a9-5a43-41a7-b797-53c006242662 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 2.672908 Oct 16 00:57:57.675528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 27/47] 10.222.0.25 () {62 vars in 1188 bytes} [Sun Oct 16 00:57:54 2022] POST /compute/v2.1/servers => generated 401 bytes in 2677 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 16 00:57:57.682512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:57.683396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:57.711641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:57.712370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:57.714353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:57.715827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:57.719468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:57.719762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:57.799970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:57.835175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c1f3c564-6bc4-49f4-acf3-8e398fd05cdb tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.927669 Oct 16 00:57:57.841427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 21/48] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:56 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 937 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:57.859065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:57.860123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:57.926060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:57.948426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e9d2cf60-4273-46a8-b62b-2dc4b8cba46a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1299 microversion: 2.1 time: 0.270193 Oct 16 00:57:57.952253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 28/49] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:57 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1299 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:57.969194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8d7ed11a-f7aa-4bed-bf41-596a5feb1c14", "tag": "37a89ecc-65c3-483a-b753-8ddbd32b4132"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:58.015917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.026285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.028449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.046095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:58.085543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:58.095643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:58.102671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-41828895-4a0c-4d30-bf8c-9023adee9ec9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1279 microversion: 2.20 time: 0.254579 Oct 16 00:57:58.107318 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 22/50] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:57:57 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1279 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:57:58.134203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:58.137808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:58.164265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.164961 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.165556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.167930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.169163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.169982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.187095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Creating event network-changed:37a89ecc-65c3-483a-b753-8ddbd32b4132 for instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 on hv-cinder-maste Oct 16 00:57:58.187566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=119012) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 16 00:57:58.187958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4926c198-560c-48a7-8363-3f0c389ca5ee req-a630131f-d9f2-4d5b-a788-5d07cad622d0 service nova] Acquired lock "compute-rpcapi-router" {{(pid=119012) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 16 00:57:58.192913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.194452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.195363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.199250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.200012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.203220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.222092 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.compute.rpcapi [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 16 00:57:58.225075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] Releasing lock "compute-rpcapi-router" {{(pid=119012) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 16 00:57:58.256715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7389ec4b-2c04-47bc-8cc2-9de5e3c807a8 None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.301781 Oct 16 00:57:58.257689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 29/51] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:57:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:57:58.603477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9dffef0-6081-49ec-9c9e-2412f35c0907", "tag": "e6727a07-61b9-4fef-83e4-902cce857bee"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:57:58.615582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.616254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.616792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.637236 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] Creating event network-changed:e6727a07-61b9-4fef-83e4-902cce857bee for instance b9dffef0-6081-49ec-9c9e-2412f35c0907 on hv-cinder-maste Oct 16 00:57:58.650181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-0811dba9-7981-4ca6-82bf-e072bf3cf67f req-44c233d6-8730-429a-b7cc-212d311f5170 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.167210 Oct 16 00:57:58.651106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 30/52] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:57:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:57:58.729465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:58.751166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7124174f-ed2a-4959-8963-aba45270fd1a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.641832 Oct 16 00:57:58.751807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 23/53] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:58 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:58.870986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:58.871847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:58.908534 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.913017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.913526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.914753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:58.915703 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:58.916512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:58.983806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:58.984339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:59.003295 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.003979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.011738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:59.013299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.013790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.014231 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:59.143396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:59.156099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a289a01d-4ed7-491d-b2e9-53111d2a6f47 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.291083 Oct 16 00:57:59.157308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 31/54] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:58 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:59.164786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:59.165588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:59.176532 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.177193 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.177689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:59.178538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.179117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.180130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:59.181177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:59.189904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-01712e1d-b6fc-450d-bde5-374a110b1808 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.217762 Oct 16 00:57:59.206319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 24/55] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:58 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:57:59.255552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:59.265481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:59.267250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:57:59.268392 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-013e695e-6a56-4d18-9e6a-43bc00010ad9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1283 microversion: 2.20 time: 0.109120 Oct 16 00:57:59.268964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 32/56] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:57:59 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1283 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:57:59.778123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:57:59.779939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:57:59.807613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.812144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.813780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:57:59.817701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:57:59.819127 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:57:59.820325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.180583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:00.181352 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:00.197793 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.198295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.198769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.199537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.200010 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.200600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.435513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.446150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-365cd5a5-3989-41e3-a740-7ad1ef52e463 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.267652 Oct 16 00:58:00.446150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 33/57] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:00 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:00.451912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:00.452774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:00.461798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.462546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.463210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.464144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.464800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.465812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.656219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.677476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-418ef275-7bbd-4f0c-8e3c-d134962b0ed1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.904653 Oct 16 00:58:00.678874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 25/58] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:57:59 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 907 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:00.697708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:00.698322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:00.708807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.712541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.715639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.716718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:00.717199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:00.717626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:00.787513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.799539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bf48bd69-deb9-4cad-8c87-730eed9532db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.353262 Oct 16 00:58:00.801412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 34/59] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:00 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:00.828817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.840729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.843043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:00.844686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1fac5a62-cc13-45e4-a321-3f409671cb70 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1331 microversion: 2.20 time: 0.163782 Oct 16 00:58:00.846069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 26/60] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:00 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1331 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:01.452790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0dbf503-273a-4834-af7e-20ee0ac8b573", "tag": "ae364987-4db0-45e9-8b32-48163c845492"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:01.466097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.466886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.467538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.468651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:01.469264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:01.471394 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=7 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:01.483463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.483970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.484415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.485130 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.485586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.486009 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.486208 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] Creating event network-changed:ae364987-4db0-45e9-8b32-48163c845492 for instance f0dbf503-273a-4834-af7e-20ee0ac8b573 on hv-cinder-maste Oct 16 00:58:01.491545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-fb6dd449-f1e6-4d37-9669-500ecf7ac834 req-0e96ee62-a9a5-422d-9c28-4cb1ec0d901e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043822 Oct 16 00:58:01.492965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 35/61] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:01.502719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=81 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:01.665191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:01.673313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-25d5bfa5-fc83-4b39-b985-de65f183567f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.209860 Oct 16 00:58:01.674199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 27/62] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:01 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:01.702814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:01.703564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:01.710275 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.711021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.711806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.713229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.714039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.714639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.827461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:01.828467 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:01.839106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.839884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.840543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.841459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.842336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.842986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.879008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:01.899997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-648d6b03-cbdc-4c7b-9fa8-0fe3721d7756 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.202077 Oct 16 00:58:01.903059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 36/63] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:01 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:01.923091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:01.923809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:01.930116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.930612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.931089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:01.931790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:01.932250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:01.932700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:02.001256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:02.010517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:02.012108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:02.013285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c0e93c5c-4ed7-491f-9711-2d7d9d96c629 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1339 microversion: 2.20 time: 0.094204 Oct 16 00:58:02.026883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 37/64] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:01 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1339 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:02.032465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:02.040573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-116409f0-a804-493e-9a79-2ac96b555cb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.217776 Oct 16 00:58:02.041991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 28/65] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:01 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:02.696873 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:02.698693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:02.707300 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:02.707978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:02.708581 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:02.709861 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:02.710471 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:02.711257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:02.927664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:02.928252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:02.941823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:02.942319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:02.942763 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:02.943566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:02.944023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:02.946268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:02.950990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:02.959048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fb0de3dd-a2c5-4bed-8efe-19918d1a20b6 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.264933 Oct 16 00:58:02.960139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 38/66] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:02 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:03.040156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:03.040838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:03.062137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:03.062796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:03.063363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:03.064166 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:03.064693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:03.065178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:03.313184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:03.328407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:03.333041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:03.334529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-cb560acc-9482-445f-ab86-3445e28dc790 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1337 microversion: 2.20 time: 0.298418 Oct 16 00:58:03.337855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 39/67] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:03 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1337 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:58:03.343055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:03.352675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:03.362333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:03.365545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:03.366189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:03.367112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:03.367716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:03.375403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:03.570118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:03.581875 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0db5a60d-8d9c-4c74-b057-6a75a512950e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.242900 Oct 16 00:58:03.583017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 40/68] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:03 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:03.988118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:03.989501 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:03.998029 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.001743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.003451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.014633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.015922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.018376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.063106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:04.071649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8183f31f-275c-4722-bd2d-ad633b2e1420 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 1.149590 Oct 16 00:58:04.072238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 29/69] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:02 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 1151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:04.268412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:04.280874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b4c3f7c0-ec99-443b-818e-cf7add986614 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.296644 Oct 16 00:58:04.292325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 41/70] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:03 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:04.378343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:04.379058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:04.396114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.397052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.398464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.410558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.411305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.412392 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.483936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a9b3320-e119-44b4-8035-2726ec9b56d4", "name": "network-vif-plugged", "status": "completed", "tag": "67ec654d-ac07-4f60-a537-631325285925"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:04.507638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.508122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.508680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.528493 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] Creating event network-vif-plugged:67ec654d-ac07-4f60-a537-631325285925 for instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 on hv-cinder-maste Oct 16 00:58:04.535121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-5d96df64-44b0-4855-a6ce-e75d04add01e req-471e29e5-3934-492b-b8f7-319cc8edfc7c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.201047 Oct 16 00:58:04.536017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 30/71] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:04.608569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:04.610320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:04.626781 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.630237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.630937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.632220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:04.632938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:04.633545 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:04.705917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:04.723591 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:04.727341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:04.734964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e222a0bb-1886-49dc-a35f-623284885ef2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1337 microversion: 2.20 time: 0.368359 Oct 16 00:58:04.750580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 42/72] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:04 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1337 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:05.097392 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:05.098956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:05.118943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.119718 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.120332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.124182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.124670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.125704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.383217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.408384 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bb633a17-40ab-4d96-b78d-343c84338a9c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.305295 Oct 16 00:58:05.410690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 43/73] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:05 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:05.421093 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.444657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-79414fda-a04f-44d8-8870-0ef085eb2843 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.842552 Oct 16 00:58:05.445834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 31/74] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:04 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 845 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:05.452679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:05.453605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:05.470401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.477872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.479609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.480730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.481583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.482316 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.745731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.756229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2b79832a-ccaf-4a2e-b51f-bf17c112a593 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.343165 Oct 16 00:58:05.756908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 44/75] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:05 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:05.766972 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:05.767589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:05.775431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.776774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.777841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.780878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:05.780878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:05.780878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:05.817853 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:05.941680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.962831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.967191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:05.970889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-de6874ed-b087-4aaf-b0dc-63cf1a49df4c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1337 microversion: 2.20 time: 0.208685 Oct 16 00:58:05.973859 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 32/76] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:05 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1337 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:06.437004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:06.437789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:06.456523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.457179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.457748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.458575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.462795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.466398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.475147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:06.476554 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:06.487695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.490654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.492091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.493021 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.493899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.494721 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.726374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:06.744304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1b59ce7b-6ae8-45ef-b6db-9c3c0f504a41 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.312507 Oct 16 00:58:06.745627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 45/77] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:06 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:06.756875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:06.769136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b8d5ac0d-b277-4314-873a-758576ef64b8 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.298356 Oct 16 00:58:06.770309 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 33/78] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:06 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:06.801153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:06.802044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:06.812200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.812941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.813546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.814422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:06.815079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:06.815654 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:06.821034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:07.000156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:07.000769 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:07.005724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:07.010119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.012418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.013228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.013969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.014441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.014886 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.015206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-806af3fa-25bd-4862-b298-b439bdfab14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.219218 Oct 16 00:58:07.015854 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 46/79] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:06 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:07.373369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8d7ed11a-f7aa-4bed-bf41-596a5feb1c14", "name": "network-vif-plugged", "status": "completed", "tag": "37a89ecc-65c3-483a-b753-8ddbd32b4132"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:07.391073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.402341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.407650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.439662 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] Creating event network-vif-plugged:37a89ecc-65c3-483a-b753-8ddbd32b4132 for instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 on hv-cinder-maste Oct 16 00:58:07.447560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:07.453628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-ba4fff49-438e-4e11-9451-deda4b4c715a req-dcee12ae-70a9-4d99-a68f-f9f13a7b0691 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082661 Oct 16 00:58:07.454273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 47/80] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:07.459654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:07.465879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:07.499790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-286089b1-9a46-479b-b0fd-0fe924325fd0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.488381 Oct 16 00:58:07.528847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 34/81] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:06 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:58:07.769172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:07.770691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:07.773215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:07.780314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.781143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.781991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.783103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.783868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.784725 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.796194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:07.797069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:07.807121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.808967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.809651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:07.812615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:07.814493 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:07.816806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.083455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=28 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:08.177250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.190006 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e44f07a1-c63d-4e87-8d32-48953a02f257 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.426012 Oct 16 00:58:08.190609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 48/82] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:07 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:08.202138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:08.205090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:08.219230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:08.221550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:08.223548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:08.224462 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.229437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:08.231108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:08.235884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.297179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.312700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f61aa99d-bd46-45db-aacb-b20462ac9540 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.522237 Oct 16 00:58:08.313913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 35/83] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:07 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:08.542339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:08.543486 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:08.554098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:08.554599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:08.555085 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.555810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:08.557135 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:08.559359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.727814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.738565 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9ac3063b-748f-4b10-b446-2dcf0ab56f11 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1315 microversion: 2.1 time: 0.546706 Oct 16 00:58:08.739836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 49/84] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:08 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1315 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:08.827821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.850614 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.857241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:08.860986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f3e2ce4d-de83-4580-a113-30e623dd7d00 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.322299 Oct 16 00:58:08.861655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:08.863393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 36/85] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:08 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:58:08.887602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:08.889078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:08.890423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:08.927340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] Creating event network-changed:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 00:58:08.942882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-7511dc2c-ac2e-4f39-a992-e71e71aa9c09 req-fe5f73ce-adb3-480d-afed-9d801a3eca5d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089099 Oct 16 00:58:08.944039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 50/86] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:58:09.220505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:09.221451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:09.244810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.257169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.257955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.259136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.261178 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.262791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.346465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:09.347622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:09.363451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.364394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.365495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.366740 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.367407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.368169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.523686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:09.532088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4bfe9783-5ab5-4106-91d3-1dc1998f1f58 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.317187 Oct 16 00:58:09.549948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 37/87] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:09 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:09.555643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9dffef0-6081-49ec-9c9e-2412f35c0907", "name": "network-vif-plugged", "status": "completed", "tag": "e6727a07-61b9-4fef-83e4-902cce857bee"}, {"server_uuid": "f0dbf503-273a-4834-af7e-20ee0ac8b573", "name": "network-vif-plugged", "status": "completed", "tag": "ae364987-4db0-45e9-8b32-48163c845492"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:09.571701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:09.577194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.578122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.578780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.581062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-290feeae-9a6a-4e38-823f-f16089e01932 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.239202 Oct 16 00:58:09.581655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 51/88] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:09 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:09.623949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Creating event network-vif-plugged:e6727a07-61b9-4fef-83e4-902cce857bee for instance b9dffef0-6081-49ec-9c9e-2412f35c0907 on hv-cinder-maste Oct 16 00:58:09.624983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] Creating event network-vif-plugged:ae364987-4db0-45e9-8b32-48163c845492 for instance f0dbf503-273a-4834-af7e-20ee0ac8b573 on hv-cinder-maste Oct 16 00:58:09.641095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-e8102295-87a2-4610-8801-bc2426ab1fb7 req-8d6a02f3-3f09-4dbf-ac7b-170e2e47de7e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.088944 Oct 16 00:58:09.644597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 38/89] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:09 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:09.764314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:09.765061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:09.771541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.772030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.772673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.773374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.773835 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.774262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.889896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:09.890710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:09.899693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.900455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.901113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.902085 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:09.902751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:09.903411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:09.951462 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:09.962893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8817153d-7457-479e-a953-ea5de5243954 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1503 microversion: 2.1 time: 0.203626 Oct 16 00:58:09.973002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 52/90] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:09 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1503 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:10.283785 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:10.293722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:10.295724 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:10.297227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-92a7ea5b-73b0-4901-96a0-62b935d35536 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.413017 Oct 16 00:58:10.311663 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 39/91] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:09 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:10.556955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:10.557936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:10.570889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:10.575292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:10.576340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:10.578893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:10.580955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:10.582510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:10.609134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:10.610625 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:10.634240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:10.640552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:10.644695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:10.647357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:10.652623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:10.659367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:10.915647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:10.931476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0488e68e-aa3a-4750-8ed6-fb6295bc76a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.376812 Oct 16 00:58:10.932490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 53/92] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:10 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:11.102338 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:11.113009 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eeea6268-2495-4165-a1ef-0976678ae190 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.508807 Oct 16 00:58:11.119918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 40/93] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:10 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:11.329295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:11.330317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:11.341051 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.341884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.342523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:11.344127 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.344798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.345689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:11.651504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:11.686618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:11.694089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:11.696094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a722ffc5-e3d7-4f20-a919-9c394be96af9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.373887 Oct 16 00:58:11.698911 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 54/94] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:11 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:11.759725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e0108aaf-441a-4a81-ba8d-9ad40eecb00e"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:11.763760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:11.774668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.775600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.776642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:11.778845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.779738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.780517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:11.963289 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:11.966044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:11.973886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.974855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.975673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:11.976821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:11.977435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:11.977997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:12.144314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:12.155258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ea93484e-516d-4d31-b8f0-f7fe6ff58945 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1322 microversion: 2.1 time: 0.201895 Oct 16 00:58:12.157062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 55/95] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:11 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1322 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:12.164592 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:12.165529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:12.177530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:12.178463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:12.179105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:12.179935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:12.180563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:12.180984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:12.382831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:12.396570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-93705fc1-e9a4-41d6-aac9-8bf0aa8262a4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1367 microversion: 2.1 time: 0.236742 Oct 16 00:58:12.397270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 56/96] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:12 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1367 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:12.733305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:12.733916 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:12.741317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:12.742008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:12.742601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:12.743568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:12.744378 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:12.745014 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:12.967219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:12.980678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:12.982578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:12.983867 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dcfd2689-cc5d-42d7-aec4-6e89043f0d57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.255133 Oct 16 00:58:13.001265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 57/97] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:12 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:58:13.182408 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:13.183563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:13.190561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.191363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.191978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.192868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.193485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.194064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.396936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:13.437546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0f50f991-bcf2-4106-a00f-7a42ff100aa6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1516 microversion: 2.1 time: 0.259256 Oct 16 00:58:13.438576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 58/98] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:13 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1516 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:13.450538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:13.451887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:13.462233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.462745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.463215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.463920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.465054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.465597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.703798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:13.716996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f52171d1-bc33-4da0-8459-aadf63c25645 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1557 microversion: 2.1 time: 0.277588 Oct 16 00:58:13.723819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 59/99] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:13 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1557 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:13.743427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:13.752903 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:13.772904 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.773399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.773836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.774540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:13.775212 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:13.775635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:13.968278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lazy-loading 'flavor' on Instance uuid b9dffef0-6081-49ec-9c9e-2412f35c0907 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 00:58:14.147317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a8e16392-1976-4f8e-b372-270fc1eb5da7 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "POST /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.394904 Oct 16 00:58:14.153628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 41/100] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 00:58:11 2022] POST /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4/os-volume_attachments => generated 194 bytes in 2403 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:14.159363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:14.159790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:14.180736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.181212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.181829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.182740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.183538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.193034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.267909 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bde4d1e7-9e7a-407d-abf6-315e966d1e93 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907/action" status: 200 len: 32 microversion: 2.1 time: 0.541155 Oct 16 00:58:14.272071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 60/101] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 00:58:13 2022] POST /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907/action => generated 32 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 00:58:14.297492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:14.299664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:14.323410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.325160 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.326425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.329434 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.331245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.332238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.625064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:14.644375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:14.647496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:14.650584 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2a195dd5-b31f-48fa-b43a-4dd95a4a3349 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.368953 Oct 16 00:58:14.652959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 61/102] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:14 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:14.662022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:14.663407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:14.676217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.677406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.678404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.681325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:14.682441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:14.683311 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:14.690755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:14.847094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a53ac262-a094-4c21-9604-0358a3ec6696 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.691210 Oct 16 00:58:14.851116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 42/103] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 00:58:14 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-interface => generated 284 bytes in 697 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:15.220938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:15.236065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-292afb23-e292-4859-839f-e47e7badf2e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.577862 Oct 16 00:58:15.237159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 62/104] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:14 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:15.399997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "name": "network-vif-plugged", "status": "completed", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:15.434052 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:15.436686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:15.440041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:15.484605 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] Creating event network-vif-plugged:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 00:58:15.502629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-d2b6ae68-600f-495c-ab32-c50b6ebeeef9 req-5a3502c7-db07-4682-8355-b3ccd3d8c73c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109501 Oct 16 00:58:15.504266 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 43/105] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:15.686453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:15.687118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:15.701791 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:15.702361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:15.702803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:15.703627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:15.704182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:15.704704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:16.264750 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:16.265962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:16.274774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:16.275806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:16.277518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:16.278924 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:16.279816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:16.280685 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:16.559932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:16.575349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:16.578492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:16.579910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:16.581965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:16.583242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-25d52125-dada-4c0b-aa1f-c94cbeb15a8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.901178 Oct 16 00:58:16.583782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 63/106] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:15 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:16.592651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8ff4e8ff-7d44-45f0-bfab-26109fff1842 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.333222 Oct 16 00:58:16.593253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 44/107] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:16 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:17.628309 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0dbf503-273a-4834-af7e-20ee0ac8b573", "tag": "ae364987-4db0-45e9-8b32-48163c845492"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:17.630105 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:17.630977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:17.641383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:17.642192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:17.643508 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:17.660710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] Creating event network-changed:ae364987-4db0-45e9-8b32-48163c845492 for instance f0dbf503-273a-4834-af7e-20ee0ac8b573 on hv-cinder-maste Oct 16 00:58:17.662448 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:17.667959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:17.668202 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-a4d71b9f-0b66-43aa-9099-cca5e47a24af req-8b07e034-39f3-44c0-9d7c-6a32cb43e7fc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048910 Oct 16 00:58:17.669139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 64/108] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:17.673762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:17.674611 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:17.675686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:17.676150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:17.677869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:17.678584 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:17.687141 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:17.687919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:17.689156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:17.690094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:17.690736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:17.691399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:17.935379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:17.952908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a7bd5017-79ba-4b80-92b6-67c5866a41bc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.282461 Oct 16 00:58:17.975457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 65/109] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:17 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:18.133115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:18.150499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:18.152978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:18.154145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c4a19940-4d8e-422b-acac-172f65f312ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1404 microversion: 2.20 time: 0.532288 Oct 16 00:58:18.159715 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 45/110] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:17 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1404 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:19.002191 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:19.005600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:19.039041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.041850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.047989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.049267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.049764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.052536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.182834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:19.184327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:19.197573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.199701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.200431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.202007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.203502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.206181 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.328734 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:19.342773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-456a7a7c-2fab-4d3a-8556-539b9973f18e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.357856 Oct 16 00:58:19.344593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 66/111] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:18 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:19.606494 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:19.615169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:19.616594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:19.617792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0ff36159-e235-400d-8459-407be9437ed9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1596 microversion: 2.20 time: 0.439568 Oct 16 00:58:19.618330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 46/112] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:58:19 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1596 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:58:19.659907 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:19.661039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:19.671214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.671887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.672901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.674078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:19.675248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:19.676200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:19.797714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1ace476f-ac5f-4bea-8a3d-02206bb722e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.147964 Oct 16 00:58:19.805967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 67/113] 10.222.0.25 () {60 vars in 1316 bytes} [Sun Oct 16 00:58:19 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-interface => generated 285 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 00:58:20.368294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:20.369180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:20.378483 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:20.379226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:20.380042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:20.381301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:20.382665 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:20.384114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:20.700225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:20.710796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8c54b122-399e-4acb-97bb-b4e8bc6a9e2c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.348318 Oct 16 00:58:20.723446 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 47/114] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:20 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:21.365165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:21.384498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:21.384982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:21.385409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:21.397537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] Creating event network-changed:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 00:58:21.411080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-e0a7bff6-e4bd-4104-953d-7339b7a8959c req-fa490b94-ef1e-4020-b37e-957f6bb6fc04 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056723 Oct 16 00:58:21.412595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 68/115] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:21.739573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:21.740341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:21.755104 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:21.757294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:21.763709 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:21.764484 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:21.765044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:21.765550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:21.953796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:21.961740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3f1125ba-f162-41e0-b84b-eba187656fd9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.225968 Oct 16 00:58:21.963587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 48/116] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:21 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:22.635641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-837076992", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:22.742271 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] validate_networks() for None {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:58:22.989673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:22.990353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:22.998724 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:22.999536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:23.000217 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:23.001114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:23.001736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:23.002387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:23.281580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:58:23.282423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:58:23.283287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:58:23.283940 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:58:23.284410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:58:23.289996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:23.297404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:23.298321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:23.298998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:23.300326 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:23.302110 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:23.302799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:23.326535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Getting quotas for project 39d1c20705604f8794fb441271e5afd3. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:58:23.332528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Getting quotas for user 86fccabd87a84389b219b00cdcc857e2 and project 39d1c20705604f8794fb441271e5afd3. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:58:23.346792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:58:23.350396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:58:23.413824 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-19acee8c-bd69-4967-80e8-c7c3cb5d2be7 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.788640 Oct 16 00:58:23.415676 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:23.419162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 69/117] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:58:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 796 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:58:23.424593 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8e410f03-e4ab-4fa5-9f38-42cd2f82d7fc tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.438730 Oct 16 00:58:23.425937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 49/118] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:22 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:23.428135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-943bdf64-d165-4a65-a65e-1ad42ddb039f tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:23.428830 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-943bdf64-d165-4a65-a65e-1ad42ddb039f tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:23.509826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-943bdf64-d165-4a65-a65e-1ad42ddb039f tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:23.518610 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-943bdf64-d165-4a65-a65e-1ad42ddb039f tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1214 microversion: 2.1 time: 0.095036 Oct 16 00:58:23.519192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 70/119] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:23 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1214 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:24.449097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:24.449812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:24.457746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:24.458238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:24.458704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:24.459440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:24.459891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:24.460497 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:24.541868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:24.542443 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:24.549090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:24.549569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:24.549995 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:24.550679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:24.551144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:24.551593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:24.602855 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:24.611162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b3fac36b-26f5-4011-a97d-62d97ef6d605 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.165916 Oct 16 00:58:24.612458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 50/120] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:24 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:24.619490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:24.627356 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3380c842-129f-4aa7-b390-9e2f2b6d89d1 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1266 microversion: 2.1 time: 0.089358 Oct 16 00:58:24.627928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 71/121] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:24 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1266 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:25.364515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:25.364998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:25.371788 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.372278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.372720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.373420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.373872 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.374300 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.401636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 00:58:25.496833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-773d8d70-6ae0-4c29-b62f-f304ac05b14b tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 204 len: 0 microversion: 2.1 time: 0.135800 Oct 16 00:58:25.497479 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 51/122] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 00:58:25 2022] DELETE /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 00:58:25.505581 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:25.506559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:25.512820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.513301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.513736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.514421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.514904 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.515337 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.634943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:25.636195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:25.642184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.643133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.643306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.645382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.645382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.645382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.645641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:25.654319 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0e76f6ce-eacd-40b9-a7bf-00ccbeb21b65 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1555 microversion: 2.1 time: 0.152437 Oct 16 00:58:25.654959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 72/123] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:25 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1555 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:25.659528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:25.660131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:25.665998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.666485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.667556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.668245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:25.668692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:25.669309 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:25.735577 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:25.743235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dd8957c5-2515-4694-a28c-41e73bc36f65 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1266 microversion: 2.1 time: 0.087183 Oct 16 00:58:25.743803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 73/124] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:25 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1266 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:25.786248 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:25.794641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f9c7e64b-b421-487f-ae5f-9a0cc6a6d0d6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.164668 Oct 16 00:58:25.795260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 52/125] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:25 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:26.677739 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:26.678377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:26.685418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.685849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.686290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.687138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.687740 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.688308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.766285 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:26.766915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:26.773890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.774382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.774815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.775555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.776014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.776503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.824048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:26.831973 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-53842dbb-b2c2-409d-94e0-373173d046cf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1555 microversion: 2.1 time: 0.158019 Oct 16 00:58:26.832558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 74/126] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:26 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1555 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:26.836989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:26.837820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:26.841599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:26.843982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.844466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.844901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.845592 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:26.846045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:26.846469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:26.849439 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2ee353c3-0e3c-4ab6-abd5-60007c928818 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1266 microversion: 2.1 time: 0.087122 Oct 16 00:58:26.850089 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 53/127] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:26 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1266 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:26.976893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:26.985197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f1d74ea7-323a-4a0f-8720-d80a53801db4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.151680 Oct 16 00:58:26.985775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 75/128] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:26 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:27.856162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:27.856797 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:27.863440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:27.863945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:27.864463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:27.865201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:27.865655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:27.866078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:27.873743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:27.874466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:27.880562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:27.881054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:27.881491 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:27.882190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:27.882647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:27.883106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:27.958053 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:27.968169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-345b160c-e06c-4b1c-ac34-f900ea2ab0ef tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1266 microversion: 2.1 time: 0.099121 Oct 16 00:58:27.968771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 76/129] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:27 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1266 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:28.008533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:28.009147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:28.011417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:28.015878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:28.016386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:28.016836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:28.017561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:28.018078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:28.018520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:28.019928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-668d2922-a3ff-4998-a77d-6ae1ab76f9e3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1555 microversion: 2.1 time: 0.167960 Oct 16 00:58:28.021918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 54/130] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:27 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1555 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:28.159926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:28.168088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2a3a11e9-4090-4523-99cf-20db4ad0426c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.163381 Oct 16 00:58:28.168660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 77/131] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:28 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:28.992067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:28.993430 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:29.000240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.000714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.001137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.001832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.002276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.002701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.044702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:29.045300 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:29.053775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.054477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.054983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.055697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.056165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.056622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.068965 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:29.077831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cb62b2cd-25ff-4e68-a1b1-c4610ddd1a46 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1266 microversion: 2.1 time: 0.089722 Oct 16 00:58:29.078557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 55/132] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:28 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1266 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:29.191969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:29.192613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:29.196665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:29.199358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.199863 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.200294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.201255 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:29.201872 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:29.202299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:29.206484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7299e23c-1957-4b7b-8e0f-120acd30539f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1555 microversion: 2.1 time: 0.165685 Oct 16 00:58:29.207111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 78/133] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:29 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1555 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:29.344762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:29.352810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-835764e9-6c4c-40a4-8c89-b275303abf5b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.164764 Oct 16 00:58:29.353429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 56/134] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:29 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:30.103349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:30.105260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:30.112721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.113350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.113989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.114964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.115772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.116420 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.204158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:30.212482 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ae781adb-464d-4b00-a5ee-1f587bc15f67 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1284 microversion: 2.1 time: 0.112872 Oct 16 00:58:30.213235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 79/135] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:30 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1284 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:30.231989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:30.232595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:30.239544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.240030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.240455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.241331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.241809 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.242234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.376308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:30.376929 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:30.383110 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.383602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.384063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.384766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:30.385224 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:30.385699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:30.527922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:30.535747 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-49c35f38-9176-47d6-96c2-60f7f263e76c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.163314 Oct 16 00:58:30.536317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 80/136] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:30 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:30.654161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:30.662524 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-fee35c2a-72d1-4560-b864-56079aa5dfdf tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1555 microversion: 2.1 time: 0.435792 Oct 16 00:58:30.663260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 57/137] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:30 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1555 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:31.237729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:31.238493 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:31.248424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.248935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.249375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.250094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.250552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.251003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.286676 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a9b3320-e119-44b4-8035-2726ec9b56d4", "name": "network-vif-deleted", "tag": "67ec654d-ac07-4f60-a537-631325285925"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:31.350112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.353221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.353921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.388473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:31.391565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] Creating event network-vif-deleted:67ec654d-ac07-4f60-a537-631325285925 for instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 on hv-cinder-maste Oct 16 00:58:31.404182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-9bc02f6e-ed0f-497b-b36b-359d25f73fdf req-9b5402b3-0b1f-4fb8-bf33-eff84004a6b4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.127604 Oct 16 00:58:31.406980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 58/138] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:58:31.408579 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f1ee6f70-689c-4ea4-ba51-03e4508267b4 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1272 microversion: 2.1 time: 0.174725 Oct 16 00:58:31.409162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 81/139] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:31 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1272 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:31.569047 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:31.582677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:31.588003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:31.606692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.608106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.609020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.624101 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.629268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.630084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.694905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:31.698735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:31.726079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.726764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.727430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.728467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:31.741126 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:31.741785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:31.748164 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:31.917305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3aa7409c9f6f46f78164818b6f6927e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c89fd0e945344feab91a37fdb93812e5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:31.952044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-13b6d4d1-cbd0-4f79-8b47-3aa3ab709010 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 200 len: 1346 microversion: 2.1 time: 0.267126 Oct 16 00:58:31.953586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 82/140] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:31 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 1346 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:31.996426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:32.004517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5d84e2ed-09b2-484e-830c-1f9ffa6672c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.447506 Oct 16 00:58:32.012509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 59/141] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:31 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:32.438268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:32.439360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:32.450332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:32.452025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:32.452710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:32.454669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:32.455365 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:32.456016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:32.655983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:32.664600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3a20c7a3-7275-4cf7-9c81-6c619555fffb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.230609 Oct 16 00:58:32.665179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 83/142] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:32 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:32.998084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:32.999246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:33.021155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.021992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.022598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.023461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.024091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.025113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.044371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] HTTP exception thrown: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Request method failure captured: Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 HTTP/1.1 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b0xTdaZrlJEEa2W6m9UAp9ZOiFbW0uu1cFOBbOO4sfq3eEIRdlvHrxtovQexVXuiI-1pfXFyThnZBCoLDXK0qOZ8BVkPHbfQhNgDj-xvAaaxkmKSPwDrZS7-Yfz3yaksdit-hby753YWLh7dNrhJQ3JVOSmUHM6qJljGEEDOn9ghA Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 00:58:33.050418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c89fd0e945344feab91a37fdb93812e5", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c89fd0e945344feab91a37fdb93812e5"}]}] Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 3aa7409c9f6f46f78164818b6f6927e4 Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.052051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '8a9b3320-e119-44b4-8035-2726ec9b56d4'} Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:58:33.053239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.054620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.056686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:33.057365 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:33.057656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Returning 404 to user: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:58:33.059131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-76424366-e407-4470-9a7f-401a723e6b2f tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 404 len: 111 microversion: 2.1 time: 0.067947 Oct 16 00:58:33.060887 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 60/143] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:32 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:58:33.078065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.078592 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.079072 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.081947 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.082699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.083648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.249325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:33.251552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:33.260022 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.261232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.261688 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.264219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.264904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.266191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.271605 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:33.288624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] HTTP exception thrown: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Request method failure captured: Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4/os-volume_attachments HTTP/1.1 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b0xTdaZrlJEEa2W6m9UAp9ZOiFbW0uu1cFOBbOO4sfq3eEIRdlvHrxtovQexVXuiI-1pfXFyThnZBCoLDXK0qOZ8BVkPHbfQhNgDj-xvAaaxkmKSPwDrZS7-Yfz3yaksdit-hby753YWLh7dNrhJQ3JVOSmUHM6qJljGEEDOn9ghA Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 00:58:33.290927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c89fd0e945344feab91a37fdb93812e5", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c89fd0e945344feab91a37fdb93812e5"}]}] Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 3aa7409c9f6f46f78164818b6f6927e4 Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.292454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'server_id': '8a9b3320-e119-44b4-8035-2726ec9b56d4'} Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:58:33.293491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:33.294850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:33.297761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Returning 404 to user: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:58:33.298884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ac998ced-3aad-4ee9-abc6-18c76dec83b3 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.054645 Oct 16 00:58:33.300502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 61/144] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 00:58:33 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4/os-volume_attachments => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 16 00:58:33.314305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:33.381964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:33.439139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-eded7d79-afaa-45c4-911c-da1670d21ac7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.351684 Oct 16 00:58:33.449113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 84/145] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:33 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:33.520695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2e7f0234-5efd-4ff6-aad5-e5bec370a100", "tag": "fbe04689-7374-4408-a0a0-f4d388dc670b"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:33.551356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.560862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.561509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.583771 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] Creating event network-changed:fbe04689-7374-4408-a0a0-f4d388dc670b for instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 on hv-cinder-maste Oct 16 00:58:33.603177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-c0d22ff0-d000-4b53-b0c7-34f177be125d req-b38bc181-a255-46f6-bb51-3e75a3e5d5a2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092558 Oct 16 00:58:33.604956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 62/146] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:33.692062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:33.693431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:33.711643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=8 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:33.717260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.718323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.720370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.722342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:33.723755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:33.726307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:33.951446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:33.959439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7dfdadd3-b6f8-4e29-ba8a-90fe11e1d73b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.272132 Oct 16 00:58:33.960021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 85/147] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:33 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:34.465800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:34.466553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:34.472586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.473066 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.473508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.474423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.474680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.475288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.626415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:34.635052 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-38e18387-2867-43f0-94ff-1e016cf0b9d3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.174108 Oct 16 00:58:34.635846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 63/148] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:34 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:34.712419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:34.713101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:34.719550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.720046 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.720512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.721214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.721673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.722099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.737168 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] HTTP exception thrown: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Request method failure captured: Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 HTTP/1.1 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b0xTdaZrlJEEa2W6m9UAp9ZOiFbW0uu1cFOBbOO4sfq3eEIRdlvHrxtovQexVXuiI-1pfXFyThnZBCoLDXK0qOZ8BVkPHbfQhNgDj-xvAaaxkmKSPwDrZS7-Yfz3yaksdit-hby753YWLh7dNrhJQ3JVOSmUHM6qJljGEEDOn9ghA Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:58:34.741974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c89fd0e945344feab91a37fdb93812e5", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c89fd0e945344feab91a37fdb93812e5"}]}] Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 3aa7409c9f6f46f78164818b6f6927e4 Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.743833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '8a9b3320-e119-44b4-8035-2726ec9b56d4'} Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:58:34.744808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.746032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.747158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Returning 404 to user: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:58:34.747158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-15be84fc-3eb7-458a-bacb-23b4ceedaae8 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 404 len: 111 microversion: 2.1 time: 0.035162 Oct 16 00:58:34.747158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 86/149] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 00:58:34 2022] DELETE /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:58:34.752501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:34.753116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] [instance: 8a9b3320-e119-44b4-8035-2726ec9b56d4] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:34.759536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.760024 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.760592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.761417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:34.761890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:34.762322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:34.777240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] HTTP exception thrown: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Request method failure captured: Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 HTTP/1.1 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b0xTdaZrlJEEa2W6m9UAp9ZOiFbW0uu1cFOBbOO4sfq3eEIRdlvHrxtovQexVXuiI-1pfXFyThnZBCoLDXK0qOZ8BVkPHbfQhNgDj-xvAaaxkmKSPwDrZS7-Yfz3yaksdit-hby753YWLh7dNrhJQ3JVOSmUHM6qJljGEEDOn9ghA Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 00:58:34.778978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c89fd0e945344feab91a37fdb93812e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c89fd0e945344feab91a37fdb93812e5", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c89fd0e945344feab91a37fdb93812e5"}]}] Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: c89fd0e945344feab91a37fdb93812e5 Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-DeleteServersTestJSON-1247050903 Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 3aa7409c9f6f46f78164818b6f6927e4 Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-DeleteServersTestJSON-1247050903-project Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.780824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '8a9b3320-e119-44b4-8035-2726ec9b56d4'} Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:58:34.781713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. Oct 16 00:58:34.783029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:58:34.784190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] Returning 404 to user: Instance 8a9b3320-e119-44b4-8035-2726ec9b56d4 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:58:34.784190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5d2327c7-aa12-43bc-b131-af2c5b07d820 tempest-DeleteServersTestJSON-1247050903 tempest-DeleteServersTestJSON-1247050903-project] 10.222.0.25 "GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4" status: 404 len: 111 microversion: 2.1 time: 0.031878 Oct 16 00:58:34.784190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 64/150] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:34 2022] GET /compute/v2.1/servers/8a9b3320-e119-44b4-8035-2726ec9b56d4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:58:34.984249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:34.985272 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:34.998666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:35.034136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.036s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:35.034903 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:35.037798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:35.039110 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:35.045364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:35.378651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:35.390200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-35c055dc-00fa-4afa-9860-b4c900b3dd41 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.412066 Oct 16 00:58:35.391299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 87/151] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:34 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:35.668209 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:35.669460 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:35.677195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:35.678475 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:35.679211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:35.680165 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:35.680933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:35.681393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:35.969221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:35.980499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3588588a-a1f8-40da-9618-9f76fe261a5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.316007 Oct 16 00:58:35.983222 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 65/152] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:35 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:36.414437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:36.415413 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:36.423869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:36.424371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:36.424872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:36.425894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:36.426360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:36.426792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:36.857669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:36.867225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-defb6363-4390-4421-82ac-a88f6fddfdc5 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.456739 Oct 16 00:58:36.867825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 88/153] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:36 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:37.007508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:37.008096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:37.032802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:37.033304 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:37.033744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:37.034472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:37.034960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:37.035386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:37.416191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:37.426440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-636d80f3-d316-4658-b4b4-b6cd9061eb7b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.422652 Oct 16 00:58:37.435033 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 66/154] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:37 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:37.897149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:37.898628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:37.914360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:37.915197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:37.915640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:37.921171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:37.921910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:37.922435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:38.186967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:38.196724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bd0920b7-7058-4b9b-8ee1-9f0d432a1828 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.305694 Oct 16 00:58:38.201599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 89/155] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:37 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:38.455774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:38.457267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:38.464787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:38.465470 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:38.466069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:38.467007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:38.469533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:38.470214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:39.034735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:39.042505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-24f31687-9492-439c-a75f-6d3b2e4e6f43 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.590421 Oct 16 00:58:39.043153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 67/156] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:38 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:39.227733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:39.228358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:39.237512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:39.238008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:39.245102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:39.246472 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:39.247160 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:39.247864 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:39.413021 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2e7f0234-5efd-4ff6-aad5-e5bec370a100", "name": "network-vif-plugged", "status": "completed", "tag": "fbe04689-7374-4408-a0a0-f4d388dc670b"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:39.445186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:39.445673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:39.446108 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:39.478444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] Creating event network-vif-plugged:fbe04689-7374-4408-a0a0-f4d388dc670b for instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 on hv-cinder-maste Oct 16 00:58:39.498708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-a66417f3-16e6-4d32-928c-57deee727ce5 req-be89f81c-6635-4179-a9a4-c634d0a56a49 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090364 Oct 16 00:58:39.501606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 68/157] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:58:39.614553 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:39.625153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-be35f124-52cb-49d7-9267-0ce6437e55ea tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.402685 Oct 16 00:58:39.630779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 90/158] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:39 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:40.076578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:40.078369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:40.087767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:40.088532 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:40.089067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:40.090237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:40.090566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:40.091212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:40.301287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:40.315878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9e199f0b-1670-4bf1-8231-2ce616959351 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.250620 Oct 16 00:58:40.317683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 69/159] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:40 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:40.649932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:40.651371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:40.657954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:40.658936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:40.659687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:40.661068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:40.661874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:40.663151 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:40.843077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:40.854031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-005984a7-4580-4ac1-b085-5e878a25cc94 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1314 microversion: 2.1 time: 0.207809 Oct 16 00:58:40.854737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 91/160] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:40 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1314 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:41.348345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:41.350100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:41.357592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:41.358207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:41.358897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:41.359818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:41.360463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:41.360891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:41.594056 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:41.604911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d1cb5f44-20fa-4481-a9db-3ab3217ff68e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.261590 Oct 16 00:58:41.606054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 70/161] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:41 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:41.877112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:41.877847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:41.883809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:41.887345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:41.888746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:41.890232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:41.891198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:41.892705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.133125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:42.148161 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b8ab15a2-9e5b-4324-b3e2-5b80f84f61cb tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1463 microversion: 2.1 time: 0.275193 Oct 16 00:58:42.148886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 92/162] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:41 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1463 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:42.208234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:42.208857 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:42.217530 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.218038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.218916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.219914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.220559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.221179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.256679 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 00:58:42.373933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-44c61ca6-6ec7-4037-b324-7a97b035ff6b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "DELETE /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 204 len: 0 microversion: 2.1 time: 0.171849 Oct 16 00:58:42.375829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 71/163] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 00:58:42 2022] DELETE /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 00:58:42.383744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:42.384516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:42.394627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.395893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.397006 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.397728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.398190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.398619 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.594439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:42.609808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-15254e9e-4431-400b-a153-2c91b1ffbd04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1469 microversion: 2.1 time: 0.229701 Oct 16 00:58:42.611626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 93/164] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:42 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1469 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:42.633088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:42.633933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:42.640313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.640815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.641256 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.641967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:42.642450 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:42.642903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:42.810310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:42.821572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f6667094-c81f-4857-876a-6abae0ebbe4a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.193052 Oct 16 00:58:42.832124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 72/165] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:42 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:43.636042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:43.637742 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:43.644771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:43.645333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:43.645840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:43.646625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:43.647182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:43.647812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:43.825291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:43.834308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-79098fa4-3a28-4f99-9313-b3e3d1f18e04 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1469 microversion: 2.1 time: 0.202490 Oct 16 00:58:43.835410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 94/166] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:43 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1469 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:43.850288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:43.851427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:43.859429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:43.860215 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:43.860739 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:43.861520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:43.862039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:43.862676 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:44.054412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:44.082050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d7852ea5-d9bc-409c-a0e2-4b2b1d6d86fb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.229053 Oct 16 00:58:44.083678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 73/167] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:43 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:44.859050 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:44.860525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:44.868099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:44.882884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:44.883992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:44.885930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:44.888799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:44.890409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:45.033817 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '86fccabd87a84389b219b00cdcc857e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39d1c20705604f8794fb441271e5afd3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:45.048949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b6d047fc-857d-4a6d-b566-0b07e2eb766b tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 200 len: 1427 microversion: 2.1 time: 0.193768 Oct 16 00:58:45.049557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 95/168] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:44 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 1427 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:45.110747 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:45.111601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:45.118794 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:45.119474 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:45.119930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:45.120627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:45.121133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:45.121887 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:45.152203 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2e7f0234-5efd-4ff6-aad5-e5bec370a100", "name": "network-vif-deleted", "tag": "fbe04689-7374-4408-a0a0-f4d388dc670b"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:58:45.169380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:45.169913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:45.170343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:45.187307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] Creating event network-vif-deleted:fbe04689-7374-4408-a0a0-f4d388dc670b for instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 on hv-cinder-maste Oct 16 00:58:45.212726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-95489b31-001e-49cb-81bd-88a5d145dde3 req-539ab0df-0066-4c1a-b7e9-e2bf8337603d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064454 Oct 16 00:58:45.214468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 96/169] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:58:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:58:45.317813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:45.326594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-175b50df-8408-47cd-baa7-e49debba01e5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.221207 Oct 16 00:58:45.327216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 74/170] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:45 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:46.077753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:46.078722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] [instance: 2e7f0234-5efd-4ff6-aad5-e5bec370a100] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:46.085784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:46.086608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:46.087361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:46.088199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:46.088886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:46.090282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:46.105186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] HTTP exception thrown: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Request method failure captured: Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 HTTP/1.1 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1csGkYMABs-Q8Wm4AZTpowf-XqGjcGSmjHG6uaN802FFnfqvS4dlbBiOIC_3lzNfBOA5kMAW23xuzWol5a6wPpyPlVNcB1SACKPbtCpEzFTHeKzcURCUomd-WxqiZmrFHylriATZ2LCtytoLY_g0TZLnckFscRz1QNwD7vSOVdxKr0 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 39d1c20705604f8794fb441271e5afd3 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesNegativeTest-476947040 Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 00:58:46.106938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/39d1c20705604f8794fb441271e5afd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/39d1c20705604f8794fb441271e5afd3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_39d1c20705604f8794fb441271e5afd3", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/39d1c20705604f8794fb441271e5afd3"}]}] Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesNegativeTest-476947040 Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 39d1c20705604f8794fb441271e5afd3 Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesNegativeTest-476947040 Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesNegativeTest-476947040-project Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 86fccabd87a84389b219b00cdcc857e2 Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesNegativeTest-476947040-project Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. Oct 16 00:58:46.108437 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '2e7f0234-5efd-4ff6-aad5-e5bec370a100'} Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:58:46.109227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. Oct 16 00:58:46.110516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 00:58:46.111762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] Returning 404 to user: Instance 2e7f0234-5efd-4ff6-aad5-e5bec370a100 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:58:46.111762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2ea49d52-3de0-460c-8b93-3b1bc6454140 tempest-VolumesNegativeTest-476947040 tempest-VolumesNegativeTest-476947040-project] 10.222.0.25 "GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100" status: 404 len: 111 microversion: 2.1 time: 0.036011 Oct 16 00:58:46.111762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 97/171] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:46 2022] GET /compute/v2.1/servers/2e7f0234-5efd-4ff6-aad5-e5bec370a100 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 00:58:46.351798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:46.353268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:46.362120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:46.363842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:46.364827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:46.366070 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:46.367050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:46.367742 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:46.540123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:46.550499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-19a666cd-6163-47bd-9ead-1ab1985ebbf6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.203999 Oct 16 00:58:46.551757 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 75/172] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:46 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:47.579766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:47.580817 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:47.588416 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:47.589940 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:47.590539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:47.591734 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:47.592556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:47.593279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:47.773561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:47.786891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a967a3c6-cc6a-442b-99c9-b7092da92e29 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.210063 Oct 16 00:58:47.787859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 98/173] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:47 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:48.828259 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:48.828873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:48.836111 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:48.836796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:48.838015 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:48.839596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:48.841692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:48.842086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:49.033631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:49.041556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-90d23f03-423c-4deb-b57e-70beb8764777 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.219985 Oct 16 00:58:49.043203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 76/174] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:48 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:50.065529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:50.066136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:50.072147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:50.072631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:50.073921 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:50.074629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:50.075130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:50.075600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:50.248510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:50.269020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d5431b77-c966-45c5-a7d0-638fdcc07bbd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.195939 Oct 16 00:58:50.270023 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 99/175] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:50 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:58:51.296387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:51.297427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:51.303950 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:51.304429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:51.305049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:51.305932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:51.306389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:51.308045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:51.468361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:51.480941 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eb01b8f4-880a-45c2-a516-d3f68a27129a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.189414 Oct 16 00:58:51.491844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 77/176] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:51 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:52.506112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:52.507333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:52.514913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:52.515421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:52.516061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:52.517082 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:52.517545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:52.518000 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:52.675534 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:52.677800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:52.687256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8d9d6c99-70ad-4774-bd6a-b4243426a01f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.184969 Oct 16 00:58:52.687843 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 100/177] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:52 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:53.713553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:53.715196 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:53.725254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:53.725968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:53.726674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:53.727856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:53.728604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:53.729311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:53.924574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:53.926915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:53.936319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9c3ae84e-bad4-4d56-963b-c03aaabd24e6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.229068 Oct 16 00:58:53.937120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 78/178] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:53 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:54.960975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:54.961589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:54.963632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:58:54.970032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:54.970519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:54.970986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:54.971690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:54.972145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:54.972570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:55.110923 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:55.119247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1ba3bc0c-c8bf-4fc9-b1b0-d850fa3efb67 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.162536 Oct 16 00:58:55.128709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 101/179] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:54 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:56.143323 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:56.143926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:56.151732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:56.152220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:56.152662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:56.153359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:56.153817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:56.154241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:56.314339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:56.325424 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4e0bdd22-6d27-4d4d-a28e-7b5481ea82c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.186282 Oct 16 00:58:56.326588 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 79/180] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:56 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:57.349938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:57.351225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:57.359213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:57.360161 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:57.360823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:57.361758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:57.362502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:57.363253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:57.506280 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:57.516203 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-630efbc9-ed17-438b-82a3-3e3792215f5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.170547 Oct 16 00:58:57.516818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 102/181] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:57 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:58.539850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:58.540386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:58.547462 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:58.547949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:58.548387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:58.549080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:58.549535 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:58.549957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:58.683923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:58.691618 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d2cc8b1d-d947-4132-aeee-083a6d25e696 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.155747 Oct 16 00:58:58.692391 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 80/182] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:58 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:58:59.717007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:58:59.717643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:58:59.723580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:59.724061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:59.724492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:59.725170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:58:59.725618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:58:59.726066 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:58:59.861274 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:58:59.869044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8e7cecad-714a-4015-bbb6-5760ffd4db4f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.156670 Oct 16 00:58:59.870114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 103/183] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:58:59 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:00.892279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:00.893907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:00.903182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:00.904384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:00.904384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:00.904776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:00.905251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:00.905649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:01.060719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:01.071619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-566ac1a7-917d-4ca9-86b5-3edd60fe038b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.183624 Oct 16 00:59:01.072225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 81/184] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:00 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:02.095250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:02.095857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:02.101820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:02.102296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:02.102729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:02.103439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:02.104622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:02.105069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:02.235846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:02.243535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5ced94e6-79a5-4be5-96e7-4679fc1e1c16 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.152176 Oct 16 00:59:02.244101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 104/185] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:02 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:03.265560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:03.266156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:03.268472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:03.273668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:03.274128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:03.274562 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:03.275272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:03.275717 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:03.276132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:03.404686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:03.412482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8ea79b9c-e15e-42e2-933b-5c609ff57973 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1522 microversion: 2.1 time: 0.150737 Oct 16 00:59:03.413049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 82/186] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:03 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:04.436637 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:04.437253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:04.443701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:04.444189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:04.444630 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:04.445324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:04.445775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:04.446199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:04.585352 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:04.593419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f9e8cf34-f4c9-49b0-a285-8799bebc4c5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1502 microversion: 2.1 time: 0.161520 Oct 16 00:59:04.593998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 105/187] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:04 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1502 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:04.617150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:04.617784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:04.625358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:04.626274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:04.626716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:04.627464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:04.627929 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:04.628608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:04.793367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:04.804119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0f27a2ff-14a4-4af5-94b4-4cf41036466e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1515 microversion: 2.1 time: 0.190674 Oct 16 00:59:04.821873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 83/188] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:04 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1515 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:09.546066 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:09.549182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:09.556772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:09.557474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:09.558078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:09.559025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:09.559622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:09.560299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:09.592766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lazy-loading 'flavor' on Instance uuid 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 00:59:09.687299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c955b675-5cc9-4434-93af-5cf0df924904 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/action" status: 200 len: 32 microversion: 2.1 time: 0.144934 Oct 16 00:59:09.688056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 106/189] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 00:59:09 2022] POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/action => generated 32 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 16 00:59:09.698170 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:09.703573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:09.714778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:09.720772 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:09.722623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:09.724235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:09.724737 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:09.726763 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:10.012840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:10.024216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-88865732-aed6-4056-a969-5733e6e8ecb4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.330521 Oct 16 00:59:10.042911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 84/190] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:09 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:11.048428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:11.049119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:11.055906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:11.056384 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:11.057474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:11.058162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:11.058609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:11.059107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:11.205891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:11.213783 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7a0b1275-30ff-4654-bd08-9db80b7326cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.169512 Oct 16 00:59:11.214397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 107/191] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:11 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:12.236855 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:12.237585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:12.244054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:12.244654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:12.245223 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:12.246086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:12.246539 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:12.246990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:12.398044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:12.411294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-49a8669b-b1d6-44d4-a18b-dd32b0472fdb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.178373 Oct 16 00:59:12.412020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 85/192] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:12 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:13.445371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:13.446529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:13.453666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:13.454455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:13.455106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:13.455798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:13.456419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:13.456986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:13.625382 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:13.634782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bd2b6b16-eb71-4841-8877-8bdcc13e9862 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.198110 Oct 16 00:59:13.635439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 108/193] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:13 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:14.662840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:14.663487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:14.673968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:14.674846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:14.675612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:14.676775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:14.677440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:14.678166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:14.850999 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:14.861398 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-dfbf0400-1cef-4bec-b60c-9561b515ba23 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.202898 Oct 16 00:59:14.877349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 86/194] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:14 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:15.886477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:15.887714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:15.895515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:15.896111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:15.896637 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:15.897455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:15.898013 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:15.898509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:16.079697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:16.089925 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d732ac4e-b991-41a1-8b71-0b6fbfa54a84 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.208901 Oct 16 00:59:16.098970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 109/195] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:15 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:17.113429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:17.114067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:17.120808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:17.121303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:17.121740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:17.122436 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:17.122990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:17.123426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:17.259972 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:17.268327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-17cbabdc-8407-48f2-90a9-46d98e7ff212 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.158505 Oct 16 00:59:17.269091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 87/196] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:17 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:18.293323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:18.293954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:18.300398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:18.300881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:18.301500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:18.302345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:18.302802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:18.303312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:18.449900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:18.456764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b32e51aa-926b-4de9-be83-cff478044f3d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.168193 Oct 16 00:59:18.457347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 110/197] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:18 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:19.481431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:19.482195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:19.488789 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:19.489298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:19.489773 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:19.490491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:19.490984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:19.491406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:19.623370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:19.632099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8d043e55-fb73-45bc-a5a4-1a84d4b69b69 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.154760 Oct 16 00:59:19.632702 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 88/198] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:19 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:20.656180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:20.656788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:20.663226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:20.663711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:20.664145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:20.664828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:20.665279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:20.665784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:20.797171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:20.807451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ed515ca4-d0ab-4dfd-807b-03c445fc25c5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.154879 Oct 16 00:59:20.808418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 111/199] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:20 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:21.352271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:21.352782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:21.359697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:21.360179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:21.360614 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:21.361305 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:21.361761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:21.362183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:21.386229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e5b4c42e-60b1-44c1-941a-8b79c436957f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/ips" status: 200 len: 150 microversion: 2.1 time: 0.037872 Oct 16 00:59:21.386804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 89/200] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 00:59:21 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/ips => generated 150 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:59:21.831519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:21.832117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:21.838189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:21.838924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:21.839512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:21.840407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:21.840917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:21.841335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:21.971723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:21.980017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1bf116e6-dc43-4527-97bd-1dec98782207 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.152346 Oct 16 00:59:21.980596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 112/201] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:21 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:23.003893 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:23.004501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:23.011914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:23.012400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:23.012837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:23.013536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:23.013987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:23.014413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:23.147235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:23.155122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-14d23fc3-9ee6-4df1-b97c-2810987d353d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.155271 Oct 16 00:59:23.155683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 90/202] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:22 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:24.179099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:24.179711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:24.185674 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:24.186165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:24.186608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:24.187409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:24.187820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:24.188256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:24.191582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:24.333453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:24.343008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-64f8940d-8545-4972-9b19-beaf64cd078c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.167614 Oct 16 00:59:24.343820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 113/203] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:24 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:25.369456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:25.370166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:25.378503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:25.379023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:25.379639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:25.380346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:25.380806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:25.381229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:25.540700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:25.549351 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a1100761-c2f4-471c-a802-1f6815c3cac5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.185272 Oct 16 00:59:25.549917 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 91/204] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:25 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:25.930394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5b358fc0-76fe-46c6-98a7-470ef9b505f8"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:25.932802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:25.934665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=26 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:25.939639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:25.940237 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:25.940667 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:25.941219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:25.941668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:25.942083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:26.229716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0266e237-b72d-4f42-8bfd-a6f5a1875e80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.303012 Oct 16 00:59:26.230509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 114/205] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 00:59:25 2022] POST /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments => generated 194 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:26.574146 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:26.574992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:26.583405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:26.583405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:26.583765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:26.584359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:26.585014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:26.585437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:26.588733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=78 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:26.764461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:26.774441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5eae7ff8-99f8-4ab8-99a8-6b99474a2223 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.205901 Oct 16 00:59:26.775508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 92/206] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:26 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:26.896208 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:26.897434 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:26.909731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:26.910548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:26.911575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:26.912851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:26.913621 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:26.914542 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:26.943401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-927dc8c2-7705-43f6-9b69-b0fd28cf9571 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/ips" status: 200 len: 152 microversion: 2.20 time: 0.055442 Oct 16 00:59:26.944034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 115/207] 10.222.0.25 () {60 vars in 1289 bytes} [Sun Oct 16 00:59:26 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/ips => generated 152 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 00:59:27.809653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:27.810288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:27.817274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:27.817766 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:27.818199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:27.819080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:27.819601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:27.819965 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:27.983164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:27.993213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-282be7b5-391d-424f-8d99-7e15a3c292ea tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.187951 Oct 16 00:59:28.008107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 93/208] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:27 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:29.017893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:29.018504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:29.026561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.027273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.027721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:29.028557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.029047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.029492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:29.176482 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:29.186660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-98cbef8e-a7b1-4953-9dc4-6ddc66d37096 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.172924 Oct 16 00:59:29.187397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 116/209] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:29 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:29.568202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:29.568811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:29.576848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.577324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.577754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:29.578443 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.578908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.579331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:29.603827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lazy-loading 'flavor' on Instance uuid d07b6833-a39f-462d-9c7e-48dda346d8cb {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 00:59:29.882716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-473f8730-cf23-4660-8d1b-60f74a9c12b9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/action" status: 202 len: 0 microversion: 2.20 time: 0.318396 Oct 16 00:59:29.883447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 94/210] 10.222.0.25 () {62 vars in 1319 bytes} [Sun Oct 16 00:59:29 2022] POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/action => generated 0 bytes in 320 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 16 00:59:29.892116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:29.892717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:29.899970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.900787 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.901612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:29.902792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:29.902792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:29.903204 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:30.051008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:30.060467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:30.061888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:30.063095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-36cd3f56-f26a-4fae-8d80-4dc99dd9627d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.174738 Oct 16 00:59:30.063657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 117/211] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:29 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:30.210825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:30.211814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:30.220066 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:30.220802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:30.221388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:30.222209 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:30.222922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:30.223411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:30.365703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:30.374534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-78308f8f-a35b-4628-95d0-4510c217f4b9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.167898 Oct 16 00:59:30.375144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 95/212] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:30 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:31.089788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:31.090530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:31.097416 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:31.098091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:31.098545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:31.099257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:31.099710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:31.100132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:31.254981 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:31.263986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:31.265942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:31.267221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-31f4089f-3a8a-4544-9152-3c0185a9917f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.182185 Oct 16 00:59:31.279320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 118/213] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:31 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:31.398191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:31.398806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:31.406917 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:31.407646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:31.408281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:31.409158 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:31.409785 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:31.410218 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:31.550681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:31.558587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1bf557dd-d981-44fa-8504-9d061bce6c9e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.164865 Oct 16 00:59:31.559464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 96/214] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:31 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:32.294629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:32.295264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:32.301614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:32.302095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:32.302528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:32.303246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:32.303700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:32.304124 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:32.582327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:32.582847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:32.590052 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:32.590560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:32.591051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:32.591752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:32.592204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:32.592632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:32.655985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:32.665223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:32.667364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:32.668642 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5b1f9986-dc7f-4e39-a67c-c4eb682653fd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.378648 Oct 16 00:59:32.670706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 119/215] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:32 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:32.735208 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:32.743006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1c67f72f-113e-467b-befd-d8414d507e05 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.164635 Oct 16 00:59:32.743577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 97/216] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:32 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:33.694613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:33.695248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:33.702755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:33.703262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:33.703694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:33.704377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:33.704824 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:33.705239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:33.766939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:33.767643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:33.771553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=32 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:33.782765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:33.783598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:33.783851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:33.785579 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:33.785579 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:33.785963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:33.929599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:33.943597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:33.945447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:33.948762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5c57379d-db52-4411-8f56-0a508137498f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.256351 Oct 16 00:59:33.949859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 120/217] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:33 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:33.957148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-91caeeea-65d5-41e5-a6fd-eaa55db46dde tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: .version_select at 0x7f8bbdb90af0>, body: {"keypair": {"name": "tempest-keypair-2093922793", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDysZMNaseUrthINBjud0Yh9j+aZRpSKT2N2fvhg7KVT1GxokWBLmG1YmYekWGhWKmks+BGFkUJruLJh+Gj0cbZWC0zK7TEN0vM67S5NjoPj96QuWnYLEjf5i3WwPcChbw=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:33.980842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-91caeeea-65d5-41e5-a6fd-eaa55db46dde tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:59:33.986722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-91caeeea-65d5-41e5-a6fd-eaa55db46dde tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:59:34.021540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-91caeeea-65d5-41e5-a6fd-eaa55db46dde tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.070068 Oct 16 00:59:34.022266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 121/218] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 00:59:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:34.027886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-497478102", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "networks": [{"uuid": "2990507f-5171-4eb8-ae44-3866cf4c43a9"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:34.031363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:34.040315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-487aaf5d-84c5-4bdc-a5bd-2a50b44c3bb0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.277808 Oct 16 00:59:34.041156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 98/219] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:33 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:34.096267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] validate_networks() for [('2990507f-5171-4eb8-ae44-3866cf4c43a9', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:59:34.581889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:59:34.581889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:59:34.583223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:59:34.583223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:59:34.583873 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:59:34.604517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:34.605862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:34.606526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:34.607564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:34.608282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:34.608836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:34.645995 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Getting quotas for project 92116166994646b39ec7a4c23e8cedc4. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:59:34.651655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Getting quotas for user e04650eb7a9349e3b4f28ded86d26b43 and project 92116166994646b39ec7a4c23e8cedc4. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:59:34.672219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:59:34.675530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:59:34.751891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4afcaac7-eccf-4de5-8b82-023450a5a8b8 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.728163 Oct 16 00:59:34.753984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 122/220] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:59:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 731 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:59:34.766050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a9070a56-4d7f-468d-a431-527ebbad9851 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:34.766671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a9070a56-4d7f-468d-a431-527ebbad9851 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:34.899335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a9070a56-4d7f-468d-a431-527ebbad9851 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:34.907782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a9070a56-4d7f-468d-a431-527ebbad9851 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1213 microversion: 2.1 time: 0.146173 Oct 16 00:59:34.910700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 99/221] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:34 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1213 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:34.981891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:34.982346 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:34.993526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:34.995367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:34.996774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:34.997836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:34.999055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:35.000842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:35.073663 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:35.074796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:35.115428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:35.116914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:35.118214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:35.122225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:35.122854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:35.123327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:35.294509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:35.313735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:35.315993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:35.317313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7ac3f22e-2dc2-4930-a18d-e98a75d062cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.344094 Oct 16 00:59:35.318352 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 123/222] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:34 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:59:35.411104 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:35.419262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c585ba23-84a7-4890-ab79-2675211a3940 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.350901 Oct 16 00:59:35.420219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 100/223] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:35 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:35.933671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:35.934302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:35.940881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:35.941676 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:35.942111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:35.942892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:35.943350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:35.943779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:35.973208 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1838909939", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7dae3d76-90a2-41d2-9dad-7f7856d5ab85"}], "security_groups": [{"name": "tempest-securitygroup--874780765"}], "key_name": "tempest-keypair-2093922793", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:36.015313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.035132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-734d7a88-65c4-4d51-944c-aa568f4a5789 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1265 microversion: 2.1 time: 0.105427 Oct 16 00:59:36.036442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 124/224] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:35 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1265 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:36.095486 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] validate_networks() for [('7dae3d76-90a2-41d2-9dad-7f7856d5ab85', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:59:36.349585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:36.350220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.356228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.356830 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.357344 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.358128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.358582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.359120 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.511989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:59:36.512223 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:59:36.512710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:59:36.513120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:59:36.513530 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:59:36.527375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.527854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.528283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.529000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.529443 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.529858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.530779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.539396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.540873 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.542130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-88776b1e-28bf-4b63-8cea-cf10d35a5eb5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.196505 Oct 16 00:59:36.547211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 125/225] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:36 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:59:36.549255 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:59:36.553985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:59:36.578813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:59:36.579388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:59:36.582899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:36.585070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.594423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.595349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.595828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.596603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.597105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.597634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.633213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f0034393-e801-4480-a2f1-6ffda2c87d24 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.664127 Oct 16 00:59:36.639008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 101/226] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:59:35 2022] POST /compute/v2.1/servers => generated 401 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 16 00:59:36.646380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-944e5dca-daa4-4575-bba1-c2ef13ae4d91 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:36.648061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-944e5dca-daa4-4575-bba1-c2ef13ae4d91 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.716449 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-944e5dca-daa4-4575-bba1-c2ef13ae4d91 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.724650 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-944e5dca-daa4-4575-bba1-c2ef13ae4d91 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1242 microversion: 2.1 time: 0.082980 Oct 16 00:59:36.739750 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 102/227] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:36 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1242 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:36.761625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:36.769446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-513d5a2c-10ac-406c-9953-39746146830e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1501 microversion: 2.1 time: 0.220658 Oct 16 00:59:36.770018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 126/228] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:36 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1501 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:36.792986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "226ba45f-bc22-428a-b29e-85e439fb5ef0"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:36.800657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.810326 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.810692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.811162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.811912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.812365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.812884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.837708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] HTTP exception thrown: Cannot 'attach_volume' instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 while it is in vm_state rescued Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Request method failure captured: Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/os-volume_attachments HTTP/1.1 Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 74 Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b-xNm9KCVjgM3wqxu3_jLffliMpX0rEBJ2pjnEEcTaHDwNHHNLws3WGBBUFguF0p4v0Uh6PcQ0OWevJVzKcLOchCd-DjlwU0EBAQ0HWFl9B8ZLgM7suC6TrG-0QTAhRmCIJzo4n5DR8fd3oJDcM_OYmQ6Ak0HH1w5jq7S_PXbqbcA Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 00:59:36.839609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5462fcbfca114c8885f77f42b479b13d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5462fcbfca114c8885f77f42b479b13d"}]}] Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: b01ce2d27bdb4c91a3f55c891f949e00 Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {"volumeAttachment": {"volumeId": "226ba45f-bc22-428a-b29e-85e439fb5ef0"}} Oct 16 00:59:36.841237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Cannot 'attach_volume' instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 while it is in vm_state rescued Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'server_id': '8d7ed11a-f7aa-4bed-bf41-596a5feb1c14', 'body': {'volumeAttachment': {'volumeId': '226ba45f-bc22-428a-b29e-85e439fb5ef0'}}} Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 00:59:36.842114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 while it is in vm_state rescued Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 00:59:36.843606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 while it is in vm_state rescued Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Returning 409 to user: Cannot 'attach_volume' instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 while it is in vm_state rescued {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-052dceff-d40a-4128-9748-ac0136bc966a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.052409 Oct 16 00:59:36.844811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 103/229] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 00:59:36 2022] POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/os-volume_attachments => generated 152 bytes in 54 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 16 00:59:36.854365 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:36.854945 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.861022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.861585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.862150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.862904 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.863397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.863819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.938143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0708806c-41ea-4d93-9970-02573d3c1358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/action" status: 202 len: 0 microversion: 2.1 time: 0.087834 Oct 16 00:59:36.942020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 127/230] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 00:59:36 2022] POST /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 00:59:36.947230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:36.948704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:36.955332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.955897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.956403 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:36.957114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:36.957567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:36.957994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.060807 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:37.061513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:37.068826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.069376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.069810 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.070756 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.071504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.072172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.171537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.180430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-08d3c3a3-5825-4031-bbc4-ad3b277d8fc9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1265 microversion: 2.1 time: 0.124674 Oct 16 00:59:37.181264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 128/231] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:37 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1265 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:37.275281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.286046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f3a86641-740c-49da-82ad-b4a35c5ac2c7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1509 microversion: 2.1 time: 0.342738 Oct 16 00:59:37.286640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 104/232] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:36 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1509 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:37.571855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:37.574324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:37.582744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.583277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.583724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.585067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.585067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.585539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.603456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:37.743666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.754689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.756252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.757100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:37.757460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-be3cc161-8095-4e4a-afca-361994172bb8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.191726 Oct 16 00:59:37.758025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 129/233] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:37 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:37.759884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:37.773090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.773575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.774007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.777603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:37.778264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:37.780294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:37.933052 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:37.941219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4c2590af-e8f4-447c-8f54-a82331885159 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1294 microversion: 2.1 time: 0.198223 Oct 16 00:59:37.942057 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 105/234] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:37 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1294 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:38.206983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:38.208443 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:38.216343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.216855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.217606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.218582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.219282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.219826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.289787 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:38.298681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bf615623-417a-47f4-b51d-68cd5585eb95 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1271 microversion: 2.1 time: 0.095972 Oct 16 00:59:38.299538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 130/235] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:38 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1271 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:38.318167 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:38.318926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:38.328490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.329209 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.329825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.330720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.331507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.332113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.587220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:38.596164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e4788550-5c28-49eb-bd47-a9742028b1d1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1509 microversion: 2.1 time: 0.284262 Oct 16 00:59:38.596766 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 106/236] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:38 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1509 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:38.789357 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:38.789967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:38.796163 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.796648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.797083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.797777 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.798226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.798641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.978260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:38.978969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:38.979227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:38.985438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.986139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.986573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.987307 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:38.987752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:38.988352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:38.988596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:38.990087 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:38.991293 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-36895e29-ebeb-464c-9ec8-610fb58f753a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.207080 Oct 16 00:59:38.991833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 131/237] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:38 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:39.090068 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:39.098300 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-273700e9-69b3-4ec0-8fd0-efd99d69f130 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1300 microversion: 2.1 time: 0.123756 Oct 16 00:59:39.099375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 107/238] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:38 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1300 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:39.324725 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:39.325588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:39.335170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:39.335661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:39.336098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:39.336798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:39.337267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:39.337708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:39.621775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:39.622770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:39.632666 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:39.633538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:39.634219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:39.635520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:39.636379 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:39.637090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:39.644102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:39.655009 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-036ffa8e-b4ce-410d-95f3-8fa422e2dfee tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.334512 Oct 16 00:59:39.668335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 132/239] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:39 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:40.021184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:40.022215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:40.030040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.030603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.031082 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.032510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.032973 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.033407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.149059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.163720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-bd464b8c-a846-4df4-8db4-a9c6f229c952 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1509 microversion: 2.1 time: 0.547011 Oct 16 00:59:40.165096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 108/240] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:39 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1509 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:40.170645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:40.171634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:40.190279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.191044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.191671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.192630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.193300 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.194224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.235321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.245492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.247844 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.249543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4adf37ff-c25b-48cc-b56d-a348cb8ad40a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.235919 Oct 16 00:59:40.251822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 133/241] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:40 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:59:40.324395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.326294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:40.334370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4b08a580-b8aa-4eee-a2b6-10d8d1522e7a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1300 microversion: 2.1 time: 0.167984 Oct 16 00:59:40.335433 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 109/242] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:40 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1300 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:40.680855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:40.682655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:40.691057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.691853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.692527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.694187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:40.694919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:40.695600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:40.918643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:40.927828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a953ef87-e0ae-4400-b06b-9358ad79a159 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.251258 Oct 16 00:59:40.936944 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 134/243] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:40 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:41.021419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13c3eeb0-533f-45a8-ba39-af5ff71addee", "tag": "d54d07bd-690c-4b92-9057-f975e6160581"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:41.067649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.068150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.068585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.111234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] Creating event network-changed:d54d07bd-690c-4b92-9057-f975e6160581 for instance 13c3eeb0-533f-45a8-ba39-af5ff71addee on hv-cinder-maste Oct 16 00:59:41.118671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-09e8d348-83de-48dd-b7ca-ce48be035b49 req-8a6730ca-ef1a-4b10-a37a-2a3e31f517b5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102175 Oct 16 00:59:41.120948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 110/244] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:41.190026 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:41.190645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:41.205652 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.206052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.206505 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.207326 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.207727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.208267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.278600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:41.279295 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:41.309741 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.311831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.312298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.318098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.318628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.322132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.455965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:41.470638 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ede63121-0891-4f7e-ba11-54e5bf4f04bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1515 microversion: 2.1 time: 0.285953 Oct 16 00:59:41.471504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 135/245] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:41 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1515 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:41.486801 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:41.487467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:41.494601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.495815 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.496613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.497781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.498576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.499273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.645526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:41.672645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:41.686337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:41.688583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7d259d7c-6e61-40df-b251-a81cc16f7acf tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.415249 Oct 16 00:59:41.689876 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 111/246] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:41 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:41.775389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:41.790585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-05f82314-3835-42eb-8d8f-fa7edc571002 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.308374 Oct 16 00:59:41.792474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 136/247] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:41 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:41.961418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:41.962041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:41.972444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.972937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.973637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:41.974577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:41.975298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:41.975936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:42.227321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:42.239645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c2033dca-cedb-45f2-a83c-35a09ae9e1ae tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.290395 Oct 16 00:59:42.241627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 112/248] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:41 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:42.716397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:42.717007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:42.723728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:42.724213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:42.724651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:42.725358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:42.725813 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:42.726263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:42.815804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:42.816412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:42.822353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:42.823673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:42.824164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:42.824822 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:42.825527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:42.826006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:42.867143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:42.876180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:42.877626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:42.878967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-81a1df0e-fa0c-4660-8b71-c39dcbaeb065 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.166979 Oct 16 00:59:42.879539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 137/249] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:42 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:43.022821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:43.027902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fc81acf6-f1bb-4184-a1b8-ca9c81e4e80a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: .version_select at 0x7f8bbddc4310>, body: {"keypair": {"name": "tempest-keypair-1657451180", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH0L3ObGlXKIjOtQr2rAtB9yTumz++Y9mHYnxZSi4SZpDzMxESZ+aEGOSsBJFIW2GNXXkdPUqHjyr1NMQDBAVwH10bqnvfIVoqWsoeS7urCD0da7p9hzP+agejk0W9+cAg=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:43.033409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a448f704-c16a-44ac-8f7b-ee0c77492ccc tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.220208 Oct 16 00:59:43.033409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 113/250] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:42 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:43.068941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-fc81acf6-f1bb-4184-a1b8-ca9c81e4e80a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for project 5462fcbfca114c8885f77f42b479b13d. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:59:43.075551 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-fc81acf6-f1bb-4184-a1b8-ca9c81e4e80a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for user b01ce2d27bdb4c91a3f55c891f949e00 and project 5462fcbfca114c8885f77f42b479b13d. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:59:43.126679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fc81acf6-f1bb-4184-a1b8-ca9c81e4e80a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.106181 Oct 16 00:59:43.127399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 138/251] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 00:59:43 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 00:59:43.144278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "18a404c6-15f6-40d3-bfbc-df301770a4fe", "tag": "4827324a-a969-452f-85b3-109a32799305"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:43.154804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:43.155316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:43.156097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:43.169504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] Creating event network-changed:4827324a-a969-452f-85b3-109a32799305 for instance 18a404c6-15f6-40d3-bfbc-df301770a4fe on hv-cinder-maste Oct 16 00:59:43.175773 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-c7224522-135d-4033-94e0-490cbef12c2d req-c4cc308e-af12-4a98-8b48-89a483b64e05 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035689 Oct 16 00:59:43.176708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 114/252] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:43.265923 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:43.266879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:43.275039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:43.276275 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:43.277041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:43.278736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:43.279468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:43.280106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:43.690582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:43.705096 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b1f0733e-04ab-41ce-b834-4394f034b00c tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.444021 Oct 16 00:59:43.707177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 139/253] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:43 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:43.906937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:43.907695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:43.922555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:43.923081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:43.923526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:43.924238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:43.924704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:43.925129 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:44.058974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:44.060291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:44.067348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:44.068211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:44.068721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:44.069499 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:44.070221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:44.071158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:44.394330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:44.417464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:44.425480 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:44.428275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-833286a4-c77f-4989-87d7-62566497cac8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1742 microversion: 2.20 time: 0.527121 Oct 16 00:59:44.449698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 115/254] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:43 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1742 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:44.465654 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:44.473589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8d0d728c-e635-40af-92d7-bab52ed3c475 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.423209 Oct 16 00:59:44.476043 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 140/255] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:44 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:44.731827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:44.732536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:44.740067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:44.740631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:44.741272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:44.742045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:44.742670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:44.743183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:44.912353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:44.920391 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1ec1cf58-c737-4252-abb9-791fd87d6e94 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.193267 Oct 16 00:59:44.921518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 116/256] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:44 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:45.459651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:45.464998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:45.468141 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:45.491860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.498072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:45.499239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:45.503562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.504384 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:45.505351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.506046 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.509418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:45.516760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.519884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.522893 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:45.528071 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.529438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.530118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:45.877234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:45.902335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:45.906441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:45.908808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5408b291-a888-4a14-a72c-13bbe46e969d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1742 microversion: 2.20 time: 0.455862 Oct 16 00:59:45.911629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 141/257] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:45 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1742 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:45.917030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:45.930468 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-66f2606a-2f1e-4f21-8c91-c7f59f59d815 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.436040 Oct 16 00:59:45.933280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 117/258] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:45 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:45.962536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:45.963055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:45.987314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.987814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.988254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:45.988977 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:45.989444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:45.989860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:46.289030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:46.303532 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-284cd1a1-a126-4359-b894-ecadaffde96a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.361414 Oct 16 00:59:46.304495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 142/259] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:45 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:46.933862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:46.934640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=10 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:46.939166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:46.940704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:46.948833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:46.949569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:46.950479 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:46.951692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:46.959247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:46.960568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:46.964716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:46.965532 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:46.975180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:46.976071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:46.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:46.977679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:46.978486 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:46.981930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:47.243479 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:47.257366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:47.259144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:47.264275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-37625306-ac57-4633-8f65-545c193aac57 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.332255 Oct 16 00:59:47.265397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 118/260] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:46 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:47.286438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1559000737", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "f6d9aa56-b330-4b62-a416-c067206c96f0"}], "security_groups": [{"name": "tempest-securitygroup--1665824567"}], "key_name": "tempest-keypair-1657451180", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:47.410962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:47.422279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bdc54d17-c7c0-4446-aca7-6edcb306696a tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.468561 Oct 16 00:59:47.427321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 143/261] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:46 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:47.439452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:47.441229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:47.458936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:47.461207 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:47.461874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:47.467372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:47.467953 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:47.468473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:47.781637 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:47.794222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-528d131f-2cec-49b5-9e93-817b2d1a2a7a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.365471 Oct 16 00:59:47.803278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 144/262] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:47 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:48.292984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:48.293624 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:48.322225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:48.327372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:48.329085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:48.333463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:48.334153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:48.341817 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:48.667073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:48.692438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:48.694892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:48.701931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d97e8ceb-f230-4654-884f-a959958b17ff tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.413244 Oct 16 00:59:48.708802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 145/263] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:48 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:59:48.732863 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:48.734515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:48.741479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:48.742802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:48.743533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:48.745125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:48.745870 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:48.747087 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:49.301105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:49.315786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-cb14e5d0-b1e9-49fa-a1bb-5c8b0ee10d9f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.605081 Oct 16 00:59:49.318970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 146/264] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:48 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:49.325957 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:49.334403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:49.346912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:49.350153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:49.350638 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:49.355651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:49.356764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:49.359057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:49.359585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:49.398025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:49.819200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:49.836102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-468977a3-3c8a-431e-9658-39e231150d57 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.515805 Oct 16 00:59:49.851743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 147/265] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:49 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:49.859049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13c3eeb0-533f-45a8-ba39-af5ff71addee", "name": "network-vif-plugged", "status": "completed", "tag": "d54d07bd-690c-4b92-9057-f975e6160581"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:49.899065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:49.899304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:49.900178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:49.959175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] Creating event network-vif-plugged:d54d07bd-690c-4b92-9057-f975e6160581 for instance 13c3eeb0-533f-45a8-ba39-af5ff71addee on hv-cinder-maste Oct 16 00:59:49.972304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-da4bb7d6-ed19-44f6-9e65-c967cab189ef req-db596b4b-389e-424f-bb12-cc4b723e3f40 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.119454 Oct 16 00:59:49.973276 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 148/266] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:49.988521 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:49.989225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:50.001755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.002247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.002687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:50.003421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.004464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.004902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:50.315702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:50.329316 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:50.332956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:50.336264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6bfb6378-ae57-4ae1-a540-efc4f6f28115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.361257 Oct 16 00:59:50.337044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 149/267] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:49 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:50.354189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:50.356214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:50.367897 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.368877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.370125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:50.371788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.372925 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.373573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:50.628288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:50.643546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c424ccfb-4e7b-4ae3-8e24-e0756b3b10df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.293714 Oct 16 00:59:50.661410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 150/268] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:50 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:50.865673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:50.866280 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:50.872865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.873363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.873796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:50.874507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:50.876755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:50.876755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:51.088100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:51.096796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8e8a682c-1820-4785-b7ee-0e7ae93b1eb4 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1313 microversion: 2.1 time: 0.235003 Oct 16 00:59:51.106924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 151/269] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:50 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1313 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:51.377969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:51.383312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:51.424186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:51.425349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:51.428512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:51.432447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:51.436827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:51.439391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:51.755016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:51.768931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:51.770399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:51.771632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3df0ac4b-25c8-4036-9645-332145fdc1c1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.410860 Oct 16 00:59:51.772188 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 152/270] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:51 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:51.788794 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] validate_networks() for [('f6d9aa56-b330-4b62-a416-c067206c96f0', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 00:59:51.791135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:51.797976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:51.819102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:51.819599 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:51.820042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:51.820755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:51.821199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:51.835789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:52.389144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:52.441347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-872706f4-f608-4f6c-898f-27c8e6340ca5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.657917 Oct 16 00:59:52.443109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 153/271] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:51 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:52.502510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "name": "network-vif-plugged", "status": "completed", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:52.528631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:52.529825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:52.530820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:52.572915 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] Creating event network-vif-plugged:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 00:59:52.592390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-4146a1a8-bd90-4fa2-a764-78ba7082cb0a req-4bff62c7-c206-453f-afa6-bb69896b74c3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124088 Oct 16 00:59:52.594454 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 154/272] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:52.610065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:52.613484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:52.631257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:52.633070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:52.634109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:52.635977 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:52.637067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:52.638181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:52.943641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 00:59:52.944396 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 00:59:52.945262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 00:59:52.945988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 00:59:52.952386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 00:59:52.975945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:52.977622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:52.979277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:52.981274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:52.982492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:52.984186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:53.034914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 00:59:53.043844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Getting quotas for user b01ce2d27bdb4c91a3f55c891f949e00 and project 5462fcbfca114c8885f77f42b479b13d. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 00:59:53.064936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 00:59:53.067732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 00:59:53.090851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=14 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:53.138649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a00ac4b2-980b-4757-89a2-cef53a28ef37 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 5.871736 Oct 16 00:59:53.139814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 119/273] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 00:59:47 2022] POST /compute/v2.1/servers => generated 409 bytes in 5874 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 00:59:53.145052 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:53.145861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:53.154012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:53.154684 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:53.156707 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:53.158777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:53.159471 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:53.161572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:53.471421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:53.490729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:53.492552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:53.497002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c50f5412-3a3c-4824-afc0-7a92d75267e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.355842 Oct 16 00:59:53.501993 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 120/274] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:53 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:53.538633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3f5c3d4f-dfa2-4247-b3cc-b2b25189d085 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:53.539287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3f5c3d4f-dfa2-4247-b3cc-b2b25189d085 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:53.595846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:53.624971 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0baf7ce3-e3a6-4082-84cf-9a1a519cef5b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1498 microversion: 2.1 time: 1.026046 Oct 16 00:59:53.626182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 155/275] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:52 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1498 bytes in 1031 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:53.654248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:53.655411 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:53.665036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:53.666261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:53.666954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:53.668248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:53.669522 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:53.671749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:53.680139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3f5c3d4f-dfa2-4247-b3cc-b2b25189d085 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:53.692083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3f5c3d4f-dfa2-4247-b3cc-b2b25189d085 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1246 microversion: 2.1 time: 0.180027 Oct 16 00:59:53.695890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 121/276] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:53 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1246 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:54.278585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:54.289331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b64dde29-779c-4da1-beef-4797da7389db tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.647923 Oct 16 00:59:54.289888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 156/277] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:53 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:54.403957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:54.405147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:54.413569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.425550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.426834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.430155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.433577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.437232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.506695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 00:59:54.529049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:54.529665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:54.544575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.545377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.547262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.548307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.562156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.564429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.743852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c8456c5a-ad68-4870-b901-17a26b61a1cc tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "DELETE /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 204 len: 0 microversion: 2.1 time: 0.344812 Oct 16 00:59:54.749766 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 122/278] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 00:59:54 2022] DELETE /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 0 bytes in 354 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 00:59:54.763674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18a404c6-15f6-40d3-bfbc-df301770a4fe", "name": "network-vif-plugged", "status": "completed", "tag": "4827324a-a969-452f-85b3-109a32799305"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:54.793933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.794411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.794892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.817710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] Creating event network-vif-plugged:4827324a-a969-452f-85b3-109a32799305 for instance 18a404c6-15f6-40d3-bfbc-df301770a4fe on hv-cinder-maste Oct 16 00:59:54.836799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-6c0e60b0-81ff-426a-900c-18db4b4e5d88 req-2556e66a-7c7f-4973-a063-6a45c0601f22 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084125 Oct 16 00:59:54.837719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 123/279] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:54.847658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:54.848644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:54.857548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.858401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.859043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.859940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:54.861106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:54.861712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:54.971286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:54.996140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-953cfc01-7b6c-4cb5-943d-6373d1d3575f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1298 microversion: 2.1 time: 0.152668 Oct 16 00:59:54.996735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 124/280] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:54 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1298 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:55.005441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:55.007543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:55.020660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:55.023912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:55.024648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:55.026307 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:55.026966 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:55.027545 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:55.044131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:55.063008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:55.064734 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:55.066930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9214081a-cc92-44a1-9d4a-92d8d0fa4c39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.542661 Oct 16 00:59:55.073367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 157/281] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:54 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 00:59:55.317570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:55.318184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:55.330354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:55.330887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:55.331331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:55.332056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:55.332514 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:55.332943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:55.377063 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:55.390191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5ae6dc01-6d3f-4f29-91cd-ed0dc6cd9e89 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1504 microversion: 2.1 time: 0.391787 Oct 16 00:59:55.393271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 125/282] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:54 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1504 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:55.774286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:55.792919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-598e368e-569b-4551-a879-477d63505b71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.478849 Oct 16 00:59:55.803858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 158/283] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:55 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:56.022261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:56.022893 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:56.042028 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:56.046547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.050228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.056011 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.060566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.063150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.077797 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.103633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:56.104468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:56.149673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.151515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.153253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.157534 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.161245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.164621 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.300657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:56.312584 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1286bd76-7766-43c2-bdf7-45c23e536daa tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1298 microversion: 2.1 time: 0.294293 Oct 16 00:59:56.314112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 126/284] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:56 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1298 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:56.423977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:56.431731 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:56.450497 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.455762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.459184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.466159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.469039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.470370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.758575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:56.772076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-af6ce540-d205-4b9e-8648-8b86d2e454b9 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1504 microversion: 2.1 time: 0.352929 Oct 16 00:59:56.778515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 127/285] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:56 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1504 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:56.830129 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:56.833464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:56.846555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.847131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.847683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.848715 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:56.849171 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:56.856062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:56.892454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 00:59:57.396125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:57.412500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0615d60-74c2-49e9-badc-dd615eeba027 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.593175 Oct 16 00:59:57.413383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 128/286] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:56 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:57.420777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:57.421757 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:57.432346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:57.433031 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:57.433644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:57.434358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:57.463668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:57.464487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.029s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:57.471162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:57.492772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:57.496142 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:57.502282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8f68284c-4e5b-433c-9497-1a69b441632a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 1.405792 Oct 16 00:59:57.517695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 159/287] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:56 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 1417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:57.666767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:57.675770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a707460c-32ce-4f3b-983f-01cfd54b2df2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1316 microversion: 2.1 time: 0.260931 Oct 16 00:59:57.679830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 129/288] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:57 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1316 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:57.803253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:57.803887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:57.818572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:57.821065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:57.823447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:57.824446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:57.828220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:57.829382 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.023183 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:58.055635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f80ba88a-f56b-4ea7-8299-9214c482afbf tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1462 microversion: 2.1 time: 0.257410 Oct 16 00:59:58.061811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 160/289] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:57 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1462 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:58.140164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13c3eeb0-533f-45a8-ba39-af5ff71addee", "name": "network-vif-deleted", "tag": "d54d07bd-690c-4b92-9057-f975e6160581"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 00:59:58.169313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.170612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.171327 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.197661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] Creating event network-vif-deleted:d54d07bd-690c-4b92-9057-f975e6160581 for instance 13c3eeb0-533f-45a8-ba39-af5ff71addee on hv-cinder-maste Oct 16 00:59:58.227205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-3cf9f7ff-fbc3-4cf2-8025-131a859990b8 req-c6d820ee-3716-4245-8c1e-76c5e6d1ac61 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091750 Oct 16 00:59:58.236206 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 130/290] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 00:59:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 00:59:58.453633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:58.453872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:58.469214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.470575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.471081 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.471853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.472424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.472746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.531560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:58.533466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:58.545994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.550330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.551093 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.553540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.555161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.557913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.803583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:58.818942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dbd89d94-45d5-4c28-a17f-1d0eadca1f94 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1367 microversion: 2.1 time: 0.369267 Oct 16 00:59:58.822750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 161/291] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:58 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1367 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 00:59:58.834108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:58.835228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:58.850070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.853470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.854492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:58.856634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:58.858433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:58.859274 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:59.280044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:59.298109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:59.303391 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:59.309284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2efb04cf-a4b3-4de8-8b78-8471cbc8fb43 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.780566 Oct 16 00:59:59.311726 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 131/292] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 00:59:58 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 00:59:59.323500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:59.327542 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:59.348932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:59.359673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:59.368290 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:59.370049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:59.372854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:59.376008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:59.461508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:59.467025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:59.467766 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:59.468393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lazy-loading 'fault' on Instance uuid 13c3eeb0-533f-45a8-ba39-af5ff71addee {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 00:59:59.581049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:59.597637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-45324b1a-7d3d-4176-a0e9-b671110cc2d7 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 200 len: 1303 microversion: 2.1 time: 0.282791 Oct 16 00:59:59.601209 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 132/293] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:59 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 1303 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:59.651383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 00:59:59.664652 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b8c69dbe-6094-4c12-b082-d667cd9e8bf3 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.840136 Oct 16 00:59:59.669999 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 162/294] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:58 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 00:59:59.857494 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 00:59:59.858622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 00:59:59.883270 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:59.883913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:59.885992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 00:59:59.891855 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 00:59:59.892594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 00:59:59.893187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.350002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:00.350627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:00.365655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.366189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.366645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.367411 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.367869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.371374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.486756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:00.502516 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eccc9d19-ab36-411d-8357-c55b53fa78dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1557 microversion: 2.1 time: 0.658668 Oct 16 01:00:00.504797 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 133/295] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 00:59:59 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1557 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:00.543456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:00.543956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:00.656384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.657991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.660583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.661748 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.662546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.663264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.899076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:00.908941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:00.913755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:00.921157 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-75743bb0-baa8-4555-a22f-6cdeb879531d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.575461 Oct 16 01:00:00.927284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 163/296] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:00 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:00.956502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:00.960048 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2627b7ab-84c0-4cb5-9f5e-4457f2c5aae3 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.420867 Oct 16 01:00:00.963806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 134/297] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:00:00 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/os-interface => generated 284 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:00.967769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 13c3eeb0-533f-45a8-ba39-af5ff71addee] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:00.969647 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:00.970987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:00.976920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.977545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.978181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.983928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.984749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.985223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.985703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.987407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.989013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:00.991291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:00.991974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:00.993028 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:01.041691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] HTTP exception thrown: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Request method failure captured: Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee HTTP/1.1 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1dz5WTrIFZ2Xr1z8eZcOzE41d7Gr0q79u9fqLX9K21VY19K1FxH9i5RaAQPjVgBstYTBLwHWkWMBfD8sZRyxARluPacDz64lbZkkrf9XVnT3gcuk420csCmU6DZD8mOcY0BY2_FUmacElf41ziy1sci7qVQ7wvaRpNS6U5AUfirqeU Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 92116166994646b39ec7a4c23e8cedc4 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:00:01.043444 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/92116166994646b39ec7a4c23e8cedc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/92116166994646b39ec7a4c23e8cedc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_92116166994646b39ec7a4c23e8cedc4", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/92116166994646b39ec7a4c23e8cedc4"}]}] Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 92116166994646b39ec7a4c23e8cedc4 Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesActionsTest-1926541035-project Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: e04650eb7a9349e3b4f28ded86d26b43 Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesActionsTest-1926541035-project Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. Oct 16 01:00:01.044823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '13c3eeb0-533f-45a8-ba39-af5ff71addee'} Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:00:01.045754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. Oct 16 01:00:01.047216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:00:01.058770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Returning 404 to user: Instance 13c3eeb0-533f-45a8-ba39-af5ff71addee could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:00:01.062807 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-faa93b88-56d9-4100-b4b0-3d2491e7211a tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee" status: 404 len: 111 microversion: 2.1 time: 0.110637 Oct 16 01:00:01.067084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 164/298] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:00 2022] GET /compute/v2.1/servers/13c3eeb0-533f-45a8-ba39-af5ff71addee => generated 111 bytes in 139 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:00:01.238407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1694233682", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "networks": [{"uuid": "2990507f-5171-4eb8-ae44-3866cf4c43a9"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:01.319435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] validate_networks() for [('2990507f-5171-4eb8-ae44-3866cf4c43a9', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:00:01.336195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:01.347862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5530b382-44a2-4516-9417-c5857e58aab0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.382698 Oct 16 01:00:01.352195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 135/299] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:00 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:01.591943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:01.615098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:01.617335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:01.618006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:01.648545 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] Creating event network-changed:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:00:01.663465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-53c31b1d-8246-4529-8730-4c9f51554d5e req-27d59681-a294-4431-bd4b-24881913ea51 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077177 Oct 16 01:00:01.667710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 136/300] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:00:01.960062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:01.960764 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:01.972143 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:01.973061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:01.973730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:01.974760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:01.975480 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:01.976147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:02.181645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:02.196409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:02.198716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:02.200602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0607001c-27bb-4690-b0ab-f63be30f43cd tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.245115 Oct 16 01:00:02.201678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 137/301] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:01 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:02.375665 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:02.376845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:02.398256 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:02.399175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:02.403243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:02.404212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:02.404870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:02.405549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:02.621025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:02.634809 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-433153a8-ac59-439e-bfbe-d3566d8022d9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.262747 Oct 16 01:00:02.636066 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 138/302] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:02 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:02.769837 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:00:02.773410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:00:02.774559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:00:02.776416 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:00:02.777106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:00:02.796944 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:02.797900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:02.798895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:02.799910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:02.800617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:02.801295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:02.866020 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Getting quotas for project 92116166994646b39ec7a4c23e8cedc4. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:00:02.875447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Getting quotas for user e04650eb7a9349e3b4f28ded86d26b43 and project 92116166994646b39ec7a4c23e8cedc4. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:00:02.893729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:00:02.897430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:00:03.034321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-296da59a-59e4-4534-bcce-c0f24ed8ff8b tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.800233 Oct 16 01:00:03.036325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 165/303] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:00:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 1803 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 16 01:00:03.059825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-edb68be7-f262-4d07-8d28-2effdc000671 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:03.060364 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-edb68be7-f262-4d07-8d28-2effdc000671 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:03.072819 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:03.198478 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-edb68be7-f262-4d07-8d28-2effdc000671 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:03.206405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-edb68be7-f262-4d07-8d28-2effdc000671 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1214 microversion: 2.1 time: 0.156197 Oct 16 01:00:03.210690 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 139/304] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:03 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1214 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:03.231068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:03.232121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:03.250894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:03.252158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:03.271224 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:03.272632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:03.274781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:03.279531 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:03.331549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:03.658314 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:03.659180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:03.677856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:03.678363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:03.680754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:03.682117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:03.682680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:03.683536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:03.844843 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:03.864312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:03.869879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:03.876626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c3211a6e-4c27-4b85-9d4f-43567d426aee tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.648957 Oct 16 01:00:03.878525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 166/305] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:03 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:04.102399 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:04.115674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a35a17c-16c0-49d3-9dfd-4920e87fbd0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.461229 Oct 16 01:00:04.132962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 140/306] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:03 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:04.229806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "18a404c6-15f6-40d3-bfbc-df301770a4fe", "tag": "4827324a-a969-452f-85b3-109a32799305"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:04.240869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:04.241415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:04.241916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:04.255099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] Creating event network-changed:4827324a-a969-452f-85b3-109a32799305 for instance 18a404c6-15f6-40d3-bfbc-df301770a4fe on hv-cinder-maste Oct 16 01:00:04.261509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-c7113c60-34a5-47f5-b0d3-9a3542c24b84 req-21da0809-d0f0-4587-bf19-1754371c6eb7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036668 Oct 16 01:00:04.265162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 141/307] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:00:04.271114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b909660e-fa1c-4801-8b7a-204c94af9927 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Action: 'create', calling method: .version_select at 0x7f8bbdbc41f0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-466177183", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMFO+9Hbzo7Yjjo5fcMG7OCqeQZ3foFPmXZP0B/r9jUPeEUzMtvM/JlRRaD99D/u1oaFoqub+WRGveHzC75A9ffYBtoQOohmxAmJHCWAVp1NMCXizqrrL+JIpU67SSfoyQ=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:04.272493 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:04.273204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:04.286332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:04.286829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:04.287941 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:04.289960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:04.291247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:04.291694 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:04.318140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b909660e-fa1c-4801-8b7a-204c94af9927 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:00:04.328847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b909660e-fa1c-4801-8b7a-204c94af9927 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for user c4e65cef89af48a7a51e5548634de75f and project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:00:04.391527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b909660e-fa1c-4801-8b7a-204c94af9927 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.511001 Oct 16 01:00:04.392881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 167/308] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:00:03 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:04.411931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:04.421356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f3813eed-a170-465a-9f9c-93ca525cabd0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1218 microversion: 2.1 time: 0.154933 Oct 16 01:00:04.440783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 142/309] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:04 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1218 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:04.906173 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:04.908376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:04.923837 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:04.925631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:04.932356 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:04.933552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:04.934292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:04.934983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:05.143658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:05.145558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:05.162783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:05.168179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:05.168886 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:05.169721 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:05.170244 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:05.170727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:05.218350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:05.232819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:05.234283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:05.242713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e068d68b-4682-4bf5-8e74-5ea3d7fd9cb3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.340769 Oct 16 01:00:05.243412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 168/310] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:04 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:05.448140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:05.448765 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:05.462626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:05.463225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:05.463680 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:05.464778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:05.465513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:05.466181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:05.502577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:05.532375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ed462e92-7857-47dd-90e0-bb28871604f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.395928 Oct 16 01:00:05.534085 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 143/311] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:05 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:05.599594 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:05.610272 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ac9db431-6957-45ba-9744-2bcbbf9ecb27 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1218 microversion: 2.1 time: 0.166384 Oct 16 01:00:05.611894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 169/312] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:05 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1218 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:06.273970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:06.275680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:06.289498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:06.290782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:06.291815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:06.292850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:06.293535 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:06.294341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:06.556875 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:06.557519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:06.666090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:06.666757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:06.674192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:06.676197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:06.684170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:06.690455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:07.033586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:07.048116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:07.072128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:07.081868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f98f9788-23c7-4224-a6b0-f1d331aa3337 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.812561 Oct 16 01:00:07.082475 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 144/313] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:06 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:07.101897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:07.110276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:07.134591 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:07.149608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:07.150768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:07.151566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:07.155722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:07.156292 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:07.212664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:07.223339 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-67cc0ddf-5692-4c89-9082-d1066e910645 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.670859 Oct 16 01:00:07.224248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 170/314] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:06 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:07.721788 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:07.740793 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-727c9c2f-5629-4cac-be05-93058b0885f6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1266 microversion: 2.1 time: 0.644523 Oct 16 01:00:07.778508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 145/315] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:07 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1266 bytes in 691 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:08.112737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:08.113466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:08.124185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.128461 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.131270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.132105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.135302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.135839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.253168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:08.253856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:08.266779 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.270386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.271899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.275154 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.275949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.278259 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.474130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:08.486661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:08.489260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:08.491401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ed61e063-9564-44fa-8fd0-3f486661b5c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.382654 Oct 16 01:00:08.492258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 171/316] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:08 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:08.795083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:08.796051 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:08.808067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.809229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.812804 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.815223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:08.819562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:08.820302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:08.932087 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:08.971537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-20b4905e-ff4c-4528-b9b6-5970b714c263 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1272 microversion: 2.1 time: 0.180191 Oct 16 01:00:08.974559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:08.988474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 172/317] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:08 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1272 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:09.029894 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a459ccb9-6c62-490a-bdd5-4465ad5e2651 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.780745 Oct 16 01:00:09.031109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 146/318] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:08 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:09.517699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:09.518294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:09.525883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:09.527801 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:09.528357 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:09.529187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:09.529716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:09.530210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:09.710603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:09.715919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "name": "network-vif-plugged", "status": "completed", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:09.725491 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:09.727069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:09.729013 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-42132978-cc2d-42e6-abad-6cde689619a7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1737 microversion: 2.20 time: 0.215488 Oct 16 01:00:09.735101 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:09.735595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 173/319] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:09 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1737 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:09.736793 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:09.737508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:09.757946 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] Creating event network-vif-plugged:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:00:09.763204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-23c0470d-a8df-47fd-a816-dc9f63f31b67 req-bbfb2602-82d5-4401-a8a2-8333c829a53b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052441 Oct 16 01:00:09.763815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 147/320] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:10.000728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:10.001719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:10.008370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.008972 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.009463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.010171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.010618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.011098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.059001 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:10.060527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:10.068481 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.069136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.069902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.070615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.071096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.071555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.163172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:10.175990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bb503fe9-439b-4471-b178-4f90c620f776 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.179705 Oct 16 01:00:10.176583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 174/321] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:09 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:10.560703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:10.563832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=100 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:10.574219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6190aac1-9b08-4a99-a3cf-15f9fa2083f4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.520052 Oct 16 01:00:10.575312 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 148/322] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:10 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:10.758031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:10.759175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:10.766291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.766828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.767414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.767986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:10.768442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:10.768865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:10.917383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:10.925985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:10.927479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:10.928658 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1316a331-82b7-49ae-9205-e2a227694015 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1719 microversion: 2.20 time: 0.174584 Oct 16 01:00:10.929258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 175/323] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:10 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1719 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:11.171820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8dd75b99-5bd7-4f32-9735-aeb21a523563", "tag": "70199f9c-b3b8-489f-9600-44bd5461aa30"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:11.183184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.183761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.184201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.200922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] Creating event network-changed:70199f9c-b3b8-489f-9600-44bd5461aa30 for instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 on hv-cinder-maste Oct 16 01:00:11.201186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:11.201887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:11.208850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-2f34bb4f-960c-4dd8-904c-f027058ed996 req-a02a9f84-7156-4227-a56b-e645b69740f7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040267 Oct 16 01:00:11.209472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 149/324] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:11.210635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.211263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.212593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.213714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.214243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.214741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.353742 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:11.362236 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3fd4e123-7f4f-44e9-a7a2-f9c0ad1c2c64 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.166325 Oct 16 01:00:11.362744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 176/325] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:11 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:11.603161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:11.603861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:11.611115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.611679 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.612190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.613025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.613600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.614100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.812662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:11.820371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2cab8d79-54bc-4873-aca4-a14bd46fc088 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1372 microversion: 2.1 time: 0.222413 Oct 16 01:00:11.820943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 150/326] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:11 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1372 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:11.956949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:11.957639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:11.964596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.965121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.965562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:11.966263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:11.966720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:11.967185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:12.109537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:12.120102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:12.124537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:12.125822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b64b4e2b-c933-4d02-9e9e-f494b8f8d1e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1719 microversion: 2.20 time: 0.173354 Oct 16 01:00:12.134641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 177/327] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:11 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1719 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:12.389835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:12.391634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:12.407749 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:12.413345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:12.413805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:12.414563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:12.415095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:12.415911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:12.660665 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:12.669423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8aa7f72e-3332-4ce8-b541-aeff01783d0e tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.286350 Oct 16 01:00:12.670007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 151/328] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:12 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:12.843703 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:12.844313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:12.846200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:12.852125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:12.852805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:12.853657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:12.854396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:12.854885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:12.855314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:12.996262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:13.004502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a73daff0-6fd6-42d9-905d-288039643a26 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1565 microversion: 2.1 time: 0.164590 Oct 16 01:00:13.005266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 178/329] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:12 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1565 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:13.028488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:13.028969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:13.035401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.036002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.036656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.037454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.037936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.038563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.140149 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-566e102a-0e5d-4b2e-9f16-775354159181 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.116285 Oct 16 01:00:13.141099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 152/330] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:00:13 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-interface => generated 284 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:13.172956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:13.173508 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:13.190638 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.191000 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.191442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.192174 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.192697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.193008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.389487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=10 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:13.403322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "name": "network-vif-unplugged", "status": "completed", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:13.420791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.421287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.421718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.436325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] Creating event network-vif-unplugged:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 01:00:13.441778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-0d3590f3-c3da-4278-959d-a8f27704ffaf req-32060d03-3bc2-41d5-a505-07f7d52c952e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.052822 Oct 16 01:00:13.442722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 153/331] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:00:13.518775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:13.533620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:13.535635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:13.537705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e6efb2ae-5320-4f85-9a97-a9865eff09e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1719 microversion: 2.20 time: 0.378159 Oct 16 01:00:13.538900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 179/332] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:13 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1719 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:00:13.694187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:13.695250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:13.705161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.706206 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.706663 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.707457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:13.707921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:13.708404 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:13.916707 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:13.932230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d6175351-f937-4c4b-b12b-799465661bd1 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.242486 Oct 16 01:00:13.945420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 154/333] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:13 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:14.569159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:14.571149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:14.584239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=10 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:14.587674 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:14.590731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:14.591864 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:14.592737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:14.594679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:14.595793 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:14.941862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:14.966815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:14.969546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:14.974748 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:14.993534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] Creating event network-changed:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:00:15.002614 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-9ef71862-0e6b-4549-9692-8791f9881854 req-c7164600-5253-422b-bcbe-b17bf813a444 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065301 Oct 16 01:00:15.003748 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 155/334] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:00:15.022662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:15.023665 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:15.032490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:15.033390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:15.040521 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:15.043996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:15.047131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:15.047861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:15.050699 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:15.052938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:15.054954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:15.056794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-51af681e-4681-489b-9ede-e3aa5c32f419 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1719 microversion: 2.20 time: 0.495755 Oct 16 01:00:15.073742 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 180/335] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:14 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1719 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:15.427913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:15.440856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2d10e8ed-485e-4642-9361-a9a2f0784335 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.435154 Oct 16 01:00:15.442040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 156/336] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:15 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:16.085739 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:16.086499 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:16.094440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.095054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.095568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.096497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.097099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.097702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.254708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:16.264692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:16.267113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:16.269027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-594c97b8-cdf6-4e51-8330-b971012103d5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1662 microversion: 2.20 time: 0.187018 Oct 16 01:00:16.281120 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 181/337] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:16 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1662 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:16.295843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f3037e6a-d239-4f27-aea5-b4ee17a7b214"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:16.298173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:16.306568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.307447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.308174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.309096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.309800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.310383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.473931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:16.474555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:16.485332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.486114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.487169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.487920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.488367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.488796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.691284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:16.700726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-139b1146-29f3-4a52-9d83-16f3d8e578af tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.232013 Oct 16 01:00:16.701323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 182/338] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:16 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:16.771550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7da751aa-b911-4841-821b-02c9512c4ae6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.479627 Oct 16 01:00:16.772442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 157/339] 10.222.0.25 () {62 vars in 1364 bytes} [Sun Oct 16 01:00:16 2022] POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments => generated 188 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 01:00:16.949684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:16.951203 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:16.961285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.962938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.964240 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:16.965121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:16.965731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:16.966305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:17.064055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119012) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 16 01:00:17.074643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:17.111636 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:17.190167 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5a875ad8-f6be-4dc9-b8e5-dbb751fb7204 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/action" status: 202 len: 0 microversion: 2.20 time: 0.245787 Oct 16 01:00:17.195412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 183/340] 10.222.0.25 () {62 vars in 1319 bytes} [Sun Oct 16 01:00:16 2022] POST /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/action => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:00:17.201707 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:17.202403 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:17.213100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:17.221569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:17.222360 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:17.223304 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:17.224109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:17.224684 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:17.725132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:17.726322 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:17.727184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:17.740419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:17.744775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:17.746340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:17.747405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-440d4f93-1623-411c-bb42-5f8092be7a6f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1748 microversion: 2.20 time: 0.550225 Oct 16 01:00:17.749889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:17.750401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:17.750733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 158/341] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:17 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1748 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:17.764070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:17.764575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:17.765031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:18.189245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:18.225359 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e96e3dfb-4e0f-43c8-9dbd-2286235356c6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.504907 Oct 16 01:00:18.229225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 184/342] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:17 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:18.781985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:18.783214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:18.791824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:18.792751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:18.794624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:18.796011 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:18.796818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:18.797441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:19.019517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:19.038589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:19.040673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:19.041865 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-40e2fb57-702c-4009-b656-930739c2abc0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1748 microversion: 2.20 time: 0.264472 Oct 16 01:00:19.042443 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 159/343] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:18 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1748 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:19.254442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:19.255510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:19.270254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:19.271034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:19.271832 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:19.278072 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:19.278566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:19.279042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:19.495974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:19.505685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-24d98d89-e5b9-4113-8f24-e3f80471e532 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.256505 Oct 16 01:00:19.506961 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 185/344] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:19 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:20.068659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:20.069992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:20.079228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:20.080128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:20.080792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:20.081848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:20.082521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:20.083187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:20.294251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:20.304762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:20.306432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:20.307568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2ed67932-3852-47b6-9a0c-bddf75f086c9 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1750 microversion: 2.20 time: 0.243126 Oct 16 01:00:20.308461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 160/345] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:20 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1750 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:20.532740 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:20.534708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:20.544673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:20.545176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:20.545618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:20.546561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:20.547600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:20.548198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:20.790003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:20.801792 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-eecc44ad-359a-48c1-b39d-e0ed88775196 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.275787 Oct 16 01:00:20.802610 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 186/346] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:20 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:21.344039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:21.345272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:21.362098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:21.364482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:21.366385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:21.367392 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:21.370288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:21.370974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:21.825859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:21.826451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:21.833795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:21.834691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:21.835417 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:21.836932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:21.837616 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:21.838264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:22.020290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:22.032671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-be5ce118-01ad-4021-aaad-5fc463048813 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.210311 Oct 16 01:00:22.033841 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 187/347] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:21 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:22.113609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8dd75b99-5bd7-4f32-9735-aeb21a523563", "name": "network-vif-plugged", "status": "completed", "tag": "70199f9c-b3b8-489f-9600-44bd5461aa30"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:22.128727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:22.129064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:22.129775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:22.155532 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] Creating event network-vif-plugged:70199f9c-b3b8-489f-9600-44bd5461aa30 for instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 on hv-cinder-maste Oct 16 01:00:22.164950 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-06dce99e-819a-46e5-9d94-bb99d38c48e3 req-ab59a50c-6f90-4413-8149-0d98abeefa94 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054567 Oct 16 01:00:22.166152 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 188/348] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:22.184952 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:22.199360 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:22.202420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:22.206070 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0fd79a2-bc20-4293-82d5-c20b2750f766 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.874238 Oct 16 01:00:22.209690 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 161/349] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:21 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:23.059015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:23.059708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:23.069058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:23.069738 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:23.070177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:23.072488 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:23.072949 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:23.073546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:23.245998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:23.246604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:23.268346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:23.269747 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:23.271453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:23.272310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:23.277506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:23.278286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:23.323579 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:23.345853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-77c9ecaa-1255-4bf0-a589-669024fc1d28 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1314 microversion: 2.1 time: 0.291529 Oct 16 01:00:23.346739 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 189/350] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:23 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1314 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:23.552721 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:23.565939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:23.567795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:23.569203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-291d0dad-0adf-4cd8-94ee-3a998cb42eb1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.334116 Oct 16 01:00:23.570254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 162/351] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:23 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:00:24.374098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:24.374725 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:24.389410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:24.389918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:24.391377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:24.392211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:24.392593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:24.395079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:24.596875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:24.599018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:24.610521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:24.620251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:24.620954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:24.621987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:24.622615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:24.623219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:24.828452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:24.841448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-aec0584a-11bc-4c85-8e25-cfedd2aa66f5 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1498 microversion: 2.1 time: 0.469719 Oct 16 01:00:24.868818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 190/352] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:24 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1498 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:24.880867 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-536205626", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-466177183", "security_groups": [{"name": "tempest-secgroup-smoke-1119436186"}], "block_device_mapping_v2": [{"uuid": "0bac125a-0773-44b5-92f7-7b1e03420312", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "67400858-429f-4052-a9a5-3012690406d5"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:25.178712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] validate_networks() for [('67400858-429f-4052-a9a5-3012690406d5', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:00:25.671906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:25.704482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:25.711775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:25.713969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-02661de2-e01e-4b61-a6e9-431cd625ed92 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 1.122478 Oct 16 01:00:25.718037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 163/353] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:24 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 1127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:25.760761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:25.761258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:25.776089 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:25.776930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:25.777534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:25.778566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:25.779252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:25.779874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:25.816581 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:00:25.880784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:25.934768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:00:25.935962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:00:25.936653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:00:25.937289 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:00:25.937885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:00:25.956571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:25.960007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:25.964072 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:25.965320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:25.966042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:25.966695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:25.978904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-42ffab8d-1345-4484-ba2b-42c90dd2e6a6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 204 len: 0 microversion: 2.1 time: 0.222115 Oct 16 01:00:25.980376 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 164/354] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:00:25 2022] DELETE /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:00:25.996656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:25.997960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:26.007041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:26.009681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:26.012703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:26.015088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:26.015921 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:00:26.016520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:26.017445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:26.028969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for user c4e65cef89af48a7a51e5548634de75f and project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:00:26.191482 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:00:26.193669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='0bac125a-0773-44b5-92f7-7b1e03420312',volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:00:26.375475 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:26.379920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fe7664fd-5f8b-46bc-ba9b-406fee43f350 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.509628 Oct 16 01:00:26.383699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 191/355] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:00:24 2022] POST /compute/v2.1/servers => generated 406 bytes in 1513 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:00:26.390339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eca9c31a-ce33-4009-853a-730fbe61c5a0 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.403675 Oct 16 01:00:26.392139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 165/356] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:25 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:26.420163 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0ed4b6d9-8421-480b-807e-6f12510e20f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:26.420861 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0ed4b6d9-8421-480b-807e-6f12510e20f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:26.509859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0ed4b6d9-8421-480b-807e-6f12510e20f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:26.524177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0ed4b6d9-8421-480b-807e-6f12510e20f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1093 microversion: 2.1 time: 0.115113 Oct 16 01:00:26.527496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 192/357] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:26 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1093 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:26.748656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:26.749647 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:26.754269 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=89,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:26.759149 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:26.759959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:26.760683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:26.761726 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:26.762576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:26.763311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:27.042706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:27.055824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:27.058396 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:27.059867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a46b60aa-0056-4c93-aed0-9bf8d0b1d783 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.317711 Oct 16 01:00:27.078437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 166/358] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:26 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:27.420684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:27.437449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:27.460361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:27.461111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:27.462406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:27.464345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:27.468465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:27.469720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:27.549283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:27.550186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:27.562243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:27.563322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:27.563949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:27.565003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:27.565645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:27.566243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:27.768665 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:27.771766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:27.779643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a80d09f-43b8-4480-a149-ea261423451b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.235214 Oct 16 01:00:27.782172 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 167/359] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:27 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:27.788869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b3b24a0a-d8e8-4775-b1a9-41bdede7de9f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.367435 Oct 16 01:00:27.790119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 193/360] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:27 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:28.085710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:28.086803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:28.094421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.094941 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.095382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.103798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.104302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.104747 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.402197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:28.416780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:28.419698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:28.421742 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-611880dc-59ad-459f-9412-52c464eecf61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.339954 Oct 16 01:00:28.436630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 168/361] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:28 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:28.797448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9432def6-4b4e-4025-93da-b2c7dc68b654 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Action: 'create', calling method: .version_select at 0x7f8bbdcaf430>, body: {"keypair": {"name": "tempest-TestShelveInstance-1829539653", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKlOPVKzjN7NMnXs/td47EpbfDiObcgAXqDV+JPOAJFgDtbrnBCKBUbfv3lp+mlYc94Y+GcufdqMR8YVjefuQq5MJdOX5yW42w17r3WBjqskp02WYI6s2Ykf3vP7UL9XtA=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:28.814047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-9432def6-4b4e-4025-93da-b2c7dc68b654 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Getting quotas for project ed24cab62e1f4810a10e7ba36fcbeadb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:00:28.818218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-9432def6-4b4e-4025-93da-b2c7dc68b654 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Getting quotas for user b899e194369a4efcb87d7cb92dc5f902 and project ed24cab62e1f4810a10e7ba36fcbeadb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:00:28.846116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9432def6-4b4e-4025-93da-b2c7dc68b654 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.240927 Oct 16 01:00:28.846762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 194/362] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:00:28 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:28.856796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:28.857410 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:28.859610 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:28.860223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:28.864017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.864726 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.865162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.865885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.866328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.866754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.868856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.869007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.869451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.870222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:28.870989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:28.871323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:28.942345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:28.954728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f5df9f98-276c-456d-acdf-beaea426df0e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.108788 Oct 16 01:00:28.956691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 169/363] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:28 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:29.096307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:29.105187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3d0dfcb5-7f72-4422-a994-2933dbd7c162 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.249663 Oct 16 01:00:29.105867 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 195/364] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:28 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:29.453643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:29.454302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:29.463266 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:29.464046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:29.464642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:29.465580 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:29.466186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:29.466760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:29.716370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:29.732436 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:29.735628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:29.737038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f3ea786b-4b53-4a71-87e7-da55fdfa8687 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.287808 Oct 16 01:00:29.737835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 170/365] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:29 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:29.999019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:29.999704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:30.015989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.016561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.017145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:30.017911 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.018367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.018888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:30.134157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:30.135132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:30.150001 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.152214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:30.154134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.154923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:30.157040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.157712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.158144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:30.161071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4cf786ed-a752-4116-9e8f-c03bb027ab2b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.166892 Oct 16 01:00:30.167638 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 196/366] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:29 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:30.457348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:30.469604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3f26bba7-d876-46a0-8df3-a399050ae6c2 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.343382 Oct 16 01:00:30.470202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 171/367] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:30 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:30.771469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:30.772865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:30.785996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.786795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.787634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:30.788571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:30.790169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:30.791364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:31.009600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:31.022573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:31.025520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:31.026795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a15e37a1-eaee-4332-b3e8-d57bd002c563 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.265900 Oct 16 01:00:31.027528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 197/368] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:30 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:31.185596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:31.187410 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:31.195995 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:31.197257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:31.198216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:31.205361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:31.206118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:31.206755 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:31.388112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:31.406265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-29983e44-040a-4239-8092-25af426c0908 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.225313 Oct 16 01:00:31.413018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 172/369] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:31 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:31.496044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:31.497102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:31.515076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:31.518691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:31.519450 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:31.520800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:31.528423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:31.531919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:31.775327 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:31.786644 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a92ababa-a659-49ad-9128-878670103aa6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.293067 Oct 16 01:00:31.787839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 198/370] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:31 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:32.054281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:32.055113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:32.065367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.065948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.066499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.067321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.067843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.068330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.263608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:32.296613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:32.298275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:32.303644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5d7e5e1b-bdc5-4c32-9f1e-41c926abef35 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.249887 Oct 16 01:00:32.305783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 173/371] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:32 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:32.442728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:32.443345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:32.456705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.457528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.457993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.458963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.459607 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.460210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.830541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:32.831504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:32.845431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.845918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.846357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.847375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:32.847990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:32.848560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:32.959917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:32.970998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8f7f4f86-e97d-4387-9036-723f5a1b1426 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.534647 Oct 16 01:00:32.972033 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 199/372] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:32 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:33.105776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:33.126007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cd19a4f6-4a6b-44eb-9361-0502ec541a15 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.311821 Oct 16 01:00:33.127005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 174/373] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:32 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:33.331716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:33.332685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:33.339465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:33.340245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:33.340868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:33.341763 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:33.342390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:33.343011 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:33.559354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:33.573496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:33.576844 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:33.578582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-66629518-45a5-434e-a6f9-c03cd0407ba7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.250935 Oct 16 01:00:33.600375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 200/374] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:33 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:33.994827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:33.995238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:34.002216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.002708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.003401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.004358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.005032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.005645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.084834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:34.093117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4a2314c3-42dd-405b-a8bb-548710c04da3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.103358 Oct 16 01:00:34.093727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 175/375] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:33 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:34.151086 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:34.152009 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:34.158340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.158840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.159324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.160083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.160494 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.160936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.313987 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:34.326366 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5406b2cf-7f7e-4cf0-9ca4-060288e2ea4f tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.180307 Oct 16 01:00:34.328674 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 201/376] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:34 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:34.611784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:34.612479 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:34.622831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.623341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.623774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.624469 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:34.625541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:34.626210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:34.781601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:34.790194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:34.791653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:34.792803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-62349a32-bc04-4127-9c82-599ba62f6228 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.185288 Oct 16 01:00:34.793358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 176/377] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:34 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:35.116093 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:35.118572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:35.123917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.124407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.124841 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.125543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.126001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.126425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.196191 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:35.207245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c78cf902-a213-4c97-8ee5-ef3bfe40d994 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1143 microversion: 2.1 time: 0.094954 Oct 16 01:00:35.207827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 202/378] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:35 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1143 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:35.352022 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:35.352799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:35.360204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.360697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.361335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.362060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.362526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.362977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.511420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:35.519344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cc9e4833-3a10-4f10-949b-16ddf8df8364 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1504 microversion: 2.1 time: 0.171803 Oct 16 01:00:35.519877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 177/379] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:35 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1504 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:35.818658 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:35.819285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:35.825513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.825984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.826405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.827130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:35.827574 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:35.827987 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:35.830634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:35.963503 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:35.972745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:35.974175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:35.975370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c6c8a3f1-8884-42a8-8f6a-3836610202e0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.160669 Oct 16 01:00:35.975955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 203/380] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:35 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:36.231166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:36.232241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:36.239254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:36.239743 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:36.240178 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:36.240884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:36.241336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:36.242301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:36.325042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:36.335093 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1c16751d-9505-402d-b4f0-9d1187fd4bf1 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1191 microversion: 2.1 time: 0.108885 Oct 16 01:00:36.336268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 178/381] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:36 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1191 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:36.543324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:36.544092 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:36.546094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:36.552668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:36.553250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:36.553882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:36.555264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:36.555879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:36.556131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8dd75b99-5bd7-4f32-9735-aeb21a523563", "name": "network-vif-deleted", "tag": "70199f9c-b3b8-489f-9600-44bd5461aa30"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:36.556672 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:36.575273 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:36.575749 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:36.576173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:36.588042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] Creating event network-vif-deleted:70199f9c-b3b8-489f-9600-44bd5461aa30 for instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 on hv-cinder-maste Oct 16 01:00:36.596504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-82592a7d-d90e-4621-a80a-c87114f2dbbf req-2a32d081-35a5-4500-99f1-689f2d9b81f2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046244 Oct 16 01:00:36.597202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 179/382] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:00:36.627783 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:36.635554 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6d25321a-f6c9-4637-b5f6-4caba75ca408 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1462 microversion: 2.1 time: 0.096663 Oct 16 01:00:36.636134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 204/383] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:36 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1462 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:37.003320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:37.004664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:37.012767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.013552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.013990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.015108 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.015632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.016301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.252044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:37.254992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:37.263445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:37.265121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:37.266408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eb12c8a8-3160-457b-8b43-657f6d416975 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.268807 Oct 16 01:00:37.266998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 180/384] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:36 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:37.362546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:37.363536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:37.371074 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.371671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.372114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.372822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.373449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.373913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.442914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:37.452541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6f2c2915-e77a-4c3d-a63e-2b83032be179 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1209 microversion: 2.1 time: 0.096019 Oct 16 01:00:37.462906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 205/385] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:37 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1209 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:37.658015 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:37.659729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:37.674089 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.674617 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.675411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.676335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.676883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.677473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.717612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:37.718358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:37.718988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:37.719599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lazy-loading 'fault' on Instance uuid 8dd75b99-5bd7-4f32-9735-aeb21a523563 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:00:37.840977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e04650eb7a9349e3b4f28ded86d26b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '92116166994646b39ec7a4c23e8cedc4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:37.859056 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-fa07202e-162a-423b-be7e-a6c968a1c157 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 200 len: 1304 microversion: 2.1 time: 0.205057 Oct 16 01:00:37.860771 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 181/386] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:37 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 1304 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:38.300304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:38.302042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:38.310688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.311344 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.312211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.313003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.313539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.314062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.479857 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:38.480820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:38.485422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:38.489632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.490203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.490708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.491517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.492600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.493106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.496614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:38.498979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:38.500414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a0b1d29d-96bb-4dea-bbf2-41d2cc7ef2f1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.206144 Oct 16 01:00:38.501150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 206/387] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:38 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:38.736368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e232da69-f419-4e6d-802e-a253ad2587ab", "tag": "1cc7e38e-e043-419a-986d-342c38dba97f"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:38.738632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:38.752435 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9b2f82f9-1311-4935-ab1a-e64f4fd6c1c5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.278804 Oct 16 01:00:38.754409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 182/388] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:38 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:38.759193 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.759733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.760172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.775771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] Creating event network-changed:1cc7e38e-e043-419a-986d-342c38dba97f for instance e232da69-f419-4e6d-802e-a253ad2587ab on hv-cinder-maste Oct 16 01:00:38.781564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-364bcc75-a985-42a8-8fd1-83a33927379a req-9cecf108-3c22-4fac-ba8e-39462440db6c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048581 Oct 16 01:00:38.782179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 207/389] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:38.889885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:38.890574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] [instance: 8dd75b99-5bd7-4f32-9735-aeb21a523563] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:38.898041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.898534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.899000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.900263 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:38.900728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:38.901180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:38.917157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] HTTP exception thrown: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Request method failure captured: Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 HTTP/1.1 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1dz5WTrIFZ2Xr1z8eZcOzE41d7Gr0q79u9fqLX9K21VY19K1FxH9i5RaAQPjVgBstYTBLwHWkWMBfD8sZRyxARluPacDz64lbZkkrf9XVnT3gcuk420csCmU6DZD8mOcY0BY2_FUmacElf41ziy1sci7qVQ7wvaRpNS6U5AUfirqeU Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 92116166994646b39ec7a4c23e8cedc4 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:00:38.918930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/92116166994646b39ec7a4c23e8cedc4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/92116166994646b39ec7a4c23e8cedc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_92116166994646b39ec7a4c23e8cedc4", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/92116166994646b39ec7a4c23e8cedc4"}]}] Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 92116166994646b39ec7a4c23e8cedc4 Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-VolumesActionsTest-1926541035 Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-VolumesActionsTest-1926541035-project Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: e04650eb7a9349e3b4f28ded86d26b43 Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-VolumesActionsTest-1926541035-project Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. Oct 16 01:00:38.920531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '8dd75b99-5bd7-4f32-9735-aeb21a523563'} Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:00:38.921680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. Oct 16 01:00:38.923164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:38.924882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] Returning 404 to user: Instance 8dd75b99-5bd7-4f32-9735-aeb21a523563 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:00:38.924882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3421ae60-df7b-4898-b93a-2f03fadd3ac6 tempest-VolumesActionsTest-1926541035 tempest-VolumesActionsTest-1926541035-project] 10.222.0.25 "GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563" status: 404 len: 111 microversion: 2.1 time: 0.035380 Oct 16 01:00:38.924882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 183/390] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:38 2022] GET /compute/v2.1/servers/8dd75b99-5bd7-4f32-9735-aeb21a523563 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:00:39.527978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:39.528711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:39.531667 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=12 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:39.536617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:39.537114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:39.537557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:39.539277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:39.539744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:39.540175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:39.693726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:39.705047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:39.706561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:39.707869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f04a4209-d9fc-4b93-8904-31bdcca347d1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.184762 Oct 16 01:00:39.708433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 208/391] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:39 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:39.779397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:39.780043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:39.787389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:39.787858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:39.788349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:39.789516 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:39.789967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:39.790383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:40.027320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:40.037466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-aa747858-bedf-4721-a5ec-fd238c13f16f tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.261889 Oct 16 01:00:40.038604 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 184/392] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:39 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:40.737813 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:40.738421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:40.745510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:40.745994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:40.746429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:40.747148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:40.747603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:40.748031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:40.884399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:40.893063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:40.894507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:40.895704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-355acaae-72b2-427c-924f-066a68df00ef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.161813 Oct 16 01:00:40.896264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 209/393] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:40 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:41.060225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:41.060826 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:41.067865 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:41.068432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:41.068926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:41.069716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:41.070175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:41.071219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:41.228832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:41.236675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f641067d-fc45-41e1-ab9e-d3e6c77c7c8c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.180611 Oct 16 01:00:41.237246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 185/394] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:41 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:41.922507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:41.923170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:41.933119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:41.933602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:41.934035 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:41.934733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:41.935221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:41.935639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:42.077499 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:42.093581 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:42.096579 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:42.098877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-83790262-5d9a-4653-98ef-adcb70a972ec tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.180443 Oct 16 01:00:42.100441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 210/395] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:41 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:42.259622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:42.260282 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:42.266510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:42.267043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:42.267485 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:42.268198 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:42.268661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:42.269386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:42.414405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:42.422990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d2805ba7-5c36-4bb8-b3ba-6b3860deb090 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.167588 Oct 16 01:00:42.423574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 186/396] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:42 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:43.128684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:43.129288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:43.136460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:43.136935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:43.137369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:43.138056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:43.138512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:43.138977 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:43.278806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:43.287951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:43.289391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:43.291349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f5d84632-6709-4351-96b8-b6b8188e65c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.166305 Oct 16 01:00:43.291935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 211/397] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:43 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:43.446008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:43.446672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:43.453671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:43.454158 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:43.454631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:43.455348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:43.455804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:43.456227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:43.591908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:43.599557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-55c3a5d7-fda5-4864-a8bc-5e7854c6b9e7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.157907 Oct 16 01:00:43.600134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 187/398] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:43 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:44.317640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:44.318244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:44.325549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:44.326033 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:44.326473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:44.327190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:44.327645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:44.328071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:44.465340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:44.475454 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:44.476881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:44.478154 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7cf30ff0-30b7-4fa2-bb07-1dbf78329d39 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.164391 Oct 16 01:00:44.478715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 212/399] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:44 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:44.622454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:44.623123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:44.629683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:44.630174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:44.630611 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:44.631382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:44.631824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:44.632304 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:44.787691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:44.797720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ef55258f-8997-4904-9d0b-c8bdcfd88081 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.179644 Oct 16 01:00:44.798684 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 188/400] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:44 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:45.504861 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:45.506328 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:45.514494 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:45.516200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:45.516750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:45.517761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:45.518358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:45.519041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:45.673578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:45.688802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:45.690617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:45.692102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5e8c397b-96c4-4609-a151-7634e8da3d90 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.191529 Oct 16 01:00:45.693223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 213/401] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:45 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:45.822586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:45.823439 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:45.830540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:45.831079 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:45.831513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:45.832418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:45.832874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:45.833300 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:46.027491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:46.037114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8fbd12aa-7459-44af-8c34-fbc1a2c5f9a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.218807 Oct 16 01:00:46.051512 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 189/402] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:45 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:46.720381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:46.722053 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:46.729719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:46.730543 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:46.731121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:46.732231 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:46.732789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:46.733448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:46.967342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:46.986542 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:46.988989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:46.990457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fc2814b5-675f-4d24-b8b6-d8102e6464f4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.274282 Oct 16 01:00:46.991517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 214/403] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:46 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:47.063514 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:47.064917 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:47.073381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.074329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.075272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.076219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.076926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.077841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.264752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:47.274335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7a16149d-133c-4268-acef-ee85b2fe6eac tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.214372 Oct 16 01:00:47.275913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 190/404] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:47 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:47.400625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:47.401278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:47.403342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=88,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:47.409097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.409755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.410191 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.410929 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.411381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.411800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.439776 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3a62bb7a-8853-4389-bc4e-078b981f97e7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/ips" status: 200 len: 150 microversion: 2.1 time: 0.044607 Oct 16 01:00:47.440325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 215/405] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:00:47 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/ips => generated 150 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:47.454440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5b358fc0-76fe-46c6-98a7-470ef9b505f8"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:47.457177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:47.463818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.464847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.464847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.466007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.467135 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.467680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.577233 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: POST /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/os-volume_attachments HTTP/1.1 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 74 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:00:47.579472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {"volumeAttachment": {"volumeId": "5b358fc0-76fe-46c6-98a7-470ef9b505f8"}} Oct 16 01:00:47.581035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'server_id': '18a404c6-15f6-40d3-bfbc-df301770a4fe', 'body': {'volumeAttachment': {'volumeId': '5b358fc0-76fe-46c6-98a7-470ef9b505f8'}}} Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:47.581832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:47.583407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 400 to user: Invalid volume: volume 5b358fc0-76fe-46c6-98a7-470ef9b505f8 is already attached to instances: f0dbf503-273a-4834-af7e-20ee0ac8b573 {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:00:47.584806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c8deb37c-0a27-4b0e-ae2e-9e53b706b63f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.131935 Oct 16 01:00:47.591690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:47.592479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:47.594811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 191/406] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:00:47 2022] POST /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe/os-volume_attachments => generated 174 bytes in 146 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:00:47.601379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.602071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.602583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.603899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.604471 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.604941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.666617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:00:47.770720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1f6ca552-4f92-4ff1-a51f-fcf1f394f8a9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 204 len: 0 microversion: 2.1 time: 0.182992 Oct 16 01:00:47.776078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 216/407] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:00:47 2022] DELETE /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:00:47.785551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:47.787898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:47.801058 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.802041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.802668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.803656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:47.804224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:47.805074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:47.967612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:47.989506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-36a1f94a-cc07-4538-897d-438c30cd1688 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1682 microversion: 2.1 time: 0.211173 Oct 16 01:00:47.992140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 192/408] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:47 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1682 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:48.018202 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:48.018959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:48.026744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:48.027349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:48.027882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:48.028675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:48.029206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:48.029860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:48.179982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:48.190063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:48.191554 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:48.192741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-26a525e0-7533-45d5-a9ea-543a4ed144bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.178938 Oct 16 01:00:48.193304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 217/409] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:48 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:48.323217 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:48.325016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:48.332869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:48.333377 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:48.333821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:48.334559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:48.335084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:48.335725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:48.491945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:48.504632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b4e8f66b-f1d3-4e34-9687-67aaac971641 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.186609 Oct 16 01:00:48.505287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 193/410] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:48 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:48.778766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1687607494", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1829539653", "security_groups": [{"name": "tempest-secgroup-smoke-65457001"}], "networks": [{"uuid": "aa50a2c7-3d8b-4cfc-9637-4222e01ba9ae"}], "block_device_mapping_v2": [{"uuid": "d3857ff0-9452-48ab-9a80-30756ef9c2c6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:48.920173 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] validate_networks() for [('aa50a2c7-3d8b-4cfc-9637-4222e01ba9ae', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:00:49.013940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:49.015019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:49.021835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.022685 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.023465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.025040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.025904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.027125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.210181 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.224451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ba47fb22-9e99-4a29-8ebe-ec15f9ad7de9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1682 microversion: 2.1 time: 0.214518 Oct 16 01:00:49.239762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 194/411] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:49 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1682 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:49.248232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:49.248849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:49.259534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.260025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.260761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.261419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.261865 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.262332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.483759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.503079 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.510591 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.511458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:00:49.512115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:00:49.512279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1b2fb503-b445-49f9-bc0e-3306eea4f3ca tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.270623 Oct 16 01:00:49.512731 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 195/412] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:49 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:49.513178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:00:49.515038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:00:49.515804 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:00:49.529572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.530302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.530959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.531902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.532551 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.533190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.549655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:49.550283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:49.554469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Getting quotas for project ed24cab62e1f4810a10e7ba36fcbeadb. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:00:49.559726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Getting quotas for user b899e194369a4efcb87d7cb92dc5f902 and project ed24cab62e1f4810a10e7ba36fcbeadb. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:00:49.565945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.569184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.569900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.579469 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:49.580316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:49.580949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:49.635742 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:00:49.638127 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='d3857ff0-9452-48ab-9a80-30756ef9c2c6',volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:00:49.807356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.816581 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c0a280c5-7520-4d49-8ba7-9dea9306cfb5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.277277 Oct 16 01:00:49.818290 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 196/413] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:49 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:49.874178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7286a57e-0cff-45ab-8984-e2a064770416 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 1.098333 Oct 16 01:00:49.875806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 218/414] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:00:48 2022] POST /compute/v2.1/servers => generated 404 bytes in 1101 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:00:49.889171 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bf43262d-b285-4160-8530-73ad0adfde8a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:49.890225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-bf43262d-b285-4160-8530-73ad0adfde8a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:49.983643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-bf43262d-b285-4160-8530-73ad0adfde8a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:49.994712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-bf43262d-b285-4160-8530-73ad0adfde8a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1089 microversion: 2.1 time: 0.111566 Oct 16 01:00:49.996204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 197/415] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:49 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1089 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:50.255700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:50.257525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:50.268092 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.268794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.269381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:50.270294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.270928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.271501 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:50.476868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:50.481018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:50.490371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-542b40d5-2064-473f-9503-14c79c7afe80 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1682 microversion: 2.1 time: 0.240894 Oct 16 01:00:50.490976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 219/416] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:50 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1682 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:50.546963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:50.547777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:50.555873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.556613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.557227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:50.558101 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.558551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.558997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:50.815850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:50.818407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=100 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:50.835507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:50.843171 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:50.844678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eb02984c-6e38-40a0-91a2-9dd4442f4caa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.301866 Oct 16 01:00:50.845382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 198/417] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:50 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:50.848694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:50.849313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:50.862583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.863791 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.880666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:50.881695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:50.882192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:50.882634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.019307 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:51.020683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:51.028760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.029518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.033569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.033569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.033984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.034635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.106078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:51.115635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-65ac9f9a-03aa-4395-b810-0dc8873d0698 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.272416 Oct 16 01:00:51.130673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 220/418] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:50 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:51.143832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:51.162430 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3b638f5d-c644-4239-8e14-f3255ff186b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1131 microversion: 2.1 time: 0.148056 Oct 16 01:00:51.174899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 199/419] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:51 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1131 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:51.521507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:51.522168 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:51.542439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.542439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.542439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.542439 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.543192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.543314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.781263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:51.790896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c74ed73d-9c4d-40c8-a97f-18e631788c7e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1682 microversion: 2.1 time: 0.280093 Oct 16 01:00:51.792616 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 221/420] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:51 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1682 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:51.872844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:51.874090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:51.889277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.890078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.890703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:51.891808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:51.892432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:51.896022 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.050659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e232da69-f419-4e6d-802e-a253ad2587ab", "name": "network-vif-plugged", "status": "completed", "tag": "1cc7e38e-e043-419a-986d-342c38dba97f"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:52.071319 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.071802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.072234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.097970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] Creating event network-vif-plugged:1cc7e38e-e043-419a-986d-342c38dba97f for instance e232da69-f419-4e6d-802e-a253ad2587ab on hv-cinder-maste Oct 16 01:00:52.108513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-37cf6555-15f0-453b-955d-ab31dd2de67d req-c98a0eea-4a06-42bf-a94e-9ad507393528 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061648 Oct 16 01:00:52.110095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 222/421] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:52.141746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:52.142449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:52.152421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.152909 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.153340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.154042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.154494 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.154932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.347519 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.378570 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.389224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.401998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9abc56ec-4659-4809-9467-ab13a253d2f2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.527184 Oct 16 01:00:52.404615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 200/422] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:51 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:52.417637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:52.418242 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:52.445706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.448234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.448859 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.455821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.458965 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.459981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.617510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.618670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.629199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b79cd7ff-b15c-4fe3-9a5b-a7e806f4d8ca tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.491739 Oct 16 01:00:52.629826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 223/423] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:52 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:52.631605 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-745f9b00-8dcd-4ae2-86fe-7848379ef6ba tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1205 microversion: 2.1 time: 0.225864 Oct 16 01:00:52.635219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 201/424] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:52 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1205 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:52.815656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:52.816409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:52.825859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.831662 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.832399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.833165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:52.833813 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:52.834422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:52.954850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:52.976396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-31f00ceb-2553-46ae-8aa5-ffde42eca4d4 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1615 microversion: 2.1 time: 0.164663 Oct 16 01:00:52.977768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 224/425] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:52 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1615 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:53.433843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:53.435488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:53.455619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.460126 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.462066 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:53.463923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.470847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.471695 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:53.653570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:53.654187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:53.667607 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.669374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.674370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:53.675445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.676081 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.677342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:53.917435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:53.927074 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b6875264-ec1c-49ab-84c1-4e6ea7646ede tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1265 microversion: 2.1 time: 0.277448 Oct 16 01:00:53.928860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 225/426] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:53 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1265 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:53.933859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:53.934840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:53.941706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.942426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.943139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:53.944028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:53.944839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:53.945409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.071620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:54.082633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c0e4ec7f-f5c1-4e2c-9390-d74c0b7a0728 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1193 microversion: 2.1 time: 0.152500 Oct 16 01:00:54.083953 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 226/427] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:53 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1193 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:54.089179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "18a404c6-15f6-40d3-bfbc-df301770a4fe", "name": "network-vif-deleted", "tag": "4827324a-a969-452f-85b3-109a32799305"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:54.107122 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.107930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.108858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.123708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] Creating event network-vif-deleted:4827324a-a969-452f-85b3-109a32799305 for instance 18a404c6-15f6-40d3-bfbc-df301770a4fe on hv-cinder-maste Oct 16 01:00:54.130362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-8cb81be2-f978-4320-8828-6e82b9a5c550 req-5265b4a3-37a7-4c97-b027-5c52a66db78b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044771 Oct 16 01:00:54.131575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 227/428] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:54.137900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:54.139292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:54.146452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.147265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.159294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.161595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.162745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.164239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.250113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.252280 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.253741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.255700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'fault' on Instance uuid 18a404c6-15f6-40d3-bfbc-df301770a4fe {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:00:54.332790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:54.345329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-799394d1-7c84-47d7-874e-a1695419fdbe tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1332 microversion: 2.1 time: 0.211623 Oct 16 01:00:54.346169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 228/429] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:54 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1332 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:54.367883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "name": "network-vif-plugged", "status": "completed", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:54.381803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.382460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.383168 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.389007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:54.402265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] Creating event network-vif-plugged:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 01:00:54.409838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-100b00e3-241f-4ba5-88e1-197400722ed3 req-c5f8a34c-7b79-4008-99f7-7213c84f2576 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048117 Oct 16 01:00:54.410896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:54.412270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 229/430] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:54.412406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:54.413495 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b2907f19-25e9-4e8a-abc1-5e9c73a22846 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.986351 Oct 16 01:00:54.414047 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 202/431] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:53 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 987 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:54.952371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:54.954545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:54.980864 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.981655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.982267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:54.987143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:54.988900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:54.990977 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.105964 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:55.106740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:55.116627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.119197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.119885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.121806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.123452 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.124168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.377840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:55.390765 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-43bb0f4c-bd8c-44a5-8bda-b949743663a0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1451 microversion: 2.1 time: 0.441839 Oct 16 01:00:55.393490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 230/432] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:54 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1451 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:55.405200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:55.411401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:55.431017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.432292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.432292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.434460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.434460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.434715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.494948 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.495622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.496220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.496853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'fault' on Instance uuid 18a404c6-15f6-40d3-bfbc-df301770a4fe {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:00:55.585190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:55.604420 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3d016be8-bd12-4e7f-96c3-2947ed4bf2b7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 200 len: 1332 microversion: 2.1 time: 0.209358 Oct 16 01:00:55.619729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 231/433] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:55 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 1332 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:55.619729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:55.619729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:55.624916 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.625424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.625867 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.626582 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.627079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.627516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.707370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:55.716858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-882003c8-fff3-4c96-8832-583dda922e30 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.615080 Oct 16 01:00:55.717442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 203/434] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:55 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:00:55.733062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:55.733848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:55.749763 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.750562 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.752536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.759204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:55.761636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:55.762666 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:55.859098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:55.867929 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2bf8633f-1a44-4a37-b940-19dc4555873e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1451 microversion: 2.1 time: 0.261828 Oct 16 01:00:55.869864 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 232/435] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:55 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1451 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:56.128814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:56.139776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:56.150592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:56.152772 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1fbd64ad-cd10-410c-8f5e-0cd7c73dd44a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.428944 Oct 16 01:00:56.156418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 204/436] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:55 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:56.350649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:56.372808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:56.373604 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:56.374461 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:56.393597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] Creating event network-changed:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:00:56.410137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-73387550-5c05-4f77-9081-eab940e8d57c req-99dfc33f-a8b1-4fc5-a372-9572f0c7d330 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063795 Oct 16 01:00:56.411803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 233/437] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:56.628793 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:56.629709 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:56.637201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:56.638036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:56.638691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:56.639882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:56.640938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:56.641939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:56.660554 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe HTTP/1.1 Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:00:56.663002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:00:56.664489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '18a404c6-15f6-40d3-bfbc-df301770a4fe'} Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:00:56.665537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:00:56.668419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:00:56.669661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:00:56.669661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7f504643-04a6-4167-853e-32c61580d640 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 404 len: 111 microversion: 2.1 time: 0.040961 Oct 16 01:00:56.669661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 205/438] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:56 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:00:56.677695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-07ad93d7-c147-4ec1-b7b8-3f7352375df0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '.version_select at 0x7f8bbdba9160>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:56.707408 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-07ad93d7-c147-4ec1-b7b8-3f7352375df0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2093922793" status: 202 len: 0 microversion: 2.1 time: 0.033916 Oct 16 01:00:56.708176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 234/439] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:00:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2093922793 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:00:56.746006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:56.746734 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:56.754103 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:56.754591 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:56.755340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:56.756806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:56.757267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:56.757872 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:57.097922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:57.107350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eefc1fcc-e4ad-43e6-85b2-d272f4bb24d5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.367431 Oct 16 01:00:57.121262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 206/440] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:56 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:57.184175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:57.187235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:57.212961 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:57.226554 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:57.228590 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:57.231992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:57.232573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:57.233016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:57.658529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:57.673761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:57.678533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:57.681046 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a7e344e7-da4d-44d0-b9ea-d87197da82e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.503128 Oct 16 01:00:57.700114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 235/441] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:57 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:00:58.135907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:58.136542 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:58.142315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 16 01:00:58.142530 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Sun Oct 16 01:00:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 (ip 10.222.0.25) !!! Oct 16 01:00:58.143525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=86 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:58.157772 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.158829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.160723 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.162247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.163061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.163719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.386096 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:58.386827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:58.401419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.401917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.407101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.409604 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.410137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.410589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.616993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'services' on Instance uuid f0dbf503-273a-4834-af7e-20ee0ac8b573 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:00:58.647682 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:58.667773 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d91b0be6-72d8-46ad-8e28-9a21b4adbbe5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.534766 Oct 16 01:00:58.671386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 207/442] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:58 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:00:58.713845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:58.714733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:58.725440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.726273 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.730579 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.731842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.732460 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.733035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.837395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5fe1b7e8-25b3-4cbb-a68f-b841d23b25dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments/5b358fc0-76fe-46c6-98a7-470ef9b505f8" status: 202 len: 0 microversion: 2.1 time: 0.457610 Oct 16 01:00:58.839044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 236/443] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:00:58 2022] DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments/5b358fc0-76fe-46c6-98a7-470ef9b505f8 => generated 0 bytes in 460 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:00:58.855228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:58.863021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:58.880905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.881799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.882459 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.883600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:58.887552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:58.888338 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:58.953010 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a48fe619-938c-41d8-8307-54b49be4d8a6 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.108294 Oct 16 01:00:58.959814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 237/444] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:00:58 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments => generated 197 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:59.144452 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:59.168488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:59.171815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:00:59.173515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-54038622-5da0-4fa3-8fdb-09e871375282 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1806 microversion: 2.20 time: 0.463886 Oct 16 01:00:59.174778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 208/445] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:00:58 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1806 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:00:59.486232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e232da69-f419-4e6d-802e-a253ad2587ab", "tag": "1cc7e38e-e043-419a-986d-342c38dba97f"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:00:59.515631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:59.517251 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:59.517687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:59.547129 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:59.551847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] Creating event network-changed:1cc7e38e-e043-419a-986d-342c38dba97f for instance e232da69-f419-4e6d-802e-a253ad2587ab on hv-cinder-maste Oct 16 01:00:59.562064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-489cfff5-5b9a-4be0-95db-ee67a55f2eeb req-016b5def-36e7-4a7a-8b18-88478d70643a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085021 Oct 16 01:00:59.563380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 238/446] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:00:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:00:59.697051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:59.697869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:59.705037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:59.706035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:59.706983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:59.708030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:59.708510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:00:59.709390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:00:59.967392 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:00:59.970877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:00:59.982040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:59.987879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:00:59.995379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:00:59.999362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.000198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.001102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.004064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.004762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.043671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:00.078349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6e88caa0-63fd-4aef-a0b4-943f8c95ef95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.115969 Oct 16 01:01:00.085317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 239/447] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:00:59 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573/os-volume_attachments => generated 25 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 16 01:01:00.101829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:00.118123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2dc3d7a5-2398-4815-9055-51b118eec1c7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.419166 Oct 16 01:01:00.119249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 209/448] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:00:59 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:00.160757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:00.162009 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:00.168339 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.169287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.169886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.171117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.171614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.172243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.204087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:00.204820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:00.219038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:01:00.219891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.220426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.221068 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.221977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.222614 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.223381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.372350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c29249d3-0869-47ed-abea-b19949990f3d tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 204 len: 0 microversion: 2.1 time: 0.214983 Oct 16 01:01:00.373347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 240/449] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:01:00 2022] DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:01:00.382490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:00.383674 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:00.391882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.392709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.393576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.395468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:00.397113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:00.397572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:00.595997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:00.601102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:00.610969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:00.611238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-aae5f9f9-d558-44eb-b29a-b0fb2cf9680e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1682 microversion: 2.1 time: 0.232883 Oct 16 01:01:00.621082 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 241/450] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:00 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1682 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:00.621329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:00.621329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-89e09cee-719d-4dd2-9984-6747c78f493e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1793 microversion: 2.20 time: 0.422437 Oct 16 01:01:00.621329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 210/451] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:00 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1793 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:01.146441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:01.147279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:01.154756 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:01.155464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:01.155950 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:01.156708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:01.157164 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:01.157619 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:01.329585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:01.340308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bdda3c9f-07d9-4068-bd52-c4b09e086287 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.198330 Oct 16 01:01:01.342902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 242/452] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:01 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:01.646462 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:01.647350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:01.654524 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:01.655380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:01.656043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:01.657017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:01.657498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:01.658141 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:01.813930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:01.825960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-62ec97b5-10fb-48cf-a2d9-5251ed95b344 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1682 microversion: 2.1 time: 0.183148 Oct 16 01:01:01.841553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 211/453] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:01 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1682 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:02.363699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:02.365171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:02.373350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:02.374196 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:02.374651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:02.375482 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:02.376109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:02.376540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:02.526114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:02.535427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fc2c450f-dee4-4efe-ad8f-32c035cd1633 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.175761 Oct 16 01:01:02.536328 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 243/454] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:02 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:02.849518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:02.850154 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:02.858086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:02.858892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:02.859513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:02.860430 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:02.861421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:02.862088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:03.144980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:03.155005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6662e50c-3ab1-4eba-905d-23ab342aa012 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1682 microversion: 2.1 time: 0.309388 Oct 16 01:01:03.156078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 212/455] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:02 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1682 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:03.559028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:03.559643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:03.567510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:03.568114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:03.568712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:03.569533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:03.570070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:03.571135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:03.688215 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0dbf503-273a-4834-af7e-20ee0ac8b573", "name": "network-vif-deleted", "tag": "ae364987-4db0-45e9-8b32-48163c845492"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:03.700200 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:03.700690 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:03.701696 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:03.717472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] Creating event network-vif-deleted:ae364987-4db0-45e9-8b32-48163c845492 for instance f0dbf503-273a-4834-af7e-20ee0ac8b573 on hv-cinder-maste Oct 16 01:01:03.723179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-2706b14a-ccd4-4503-9ce5-2c283ec02167 req-46029a83-b6f9-45b6-8e41-7f075776b116 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038440 Oct 16 01:01:03.723833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 213/456] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:01:03.809084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:03.817431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-cd0a4c25-4896-42df-9f55-ea50c0f9e6ab tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.262427 Oct 16 01:01:03.818019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 244/457] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:03 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:04.180492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:04.181230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:04.198818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:04.199974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:04.203881 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:04.205061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:04.206555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:04.207049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:04.283302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:04.294311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f1da2e78-be92-4d8b-aa71-dd3571cdee95 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 200 len: 1326 microversion: 2.1 time: 0.119668 Oct 16 01:01:04.295126 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 214/458] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:04 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 1326 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:04.841313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:04.842135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:04.849140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:04.849715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:04.850213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:04.850974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:04.851481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:04.852127 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:04.986609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:04.995115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-09a8c4a0-272e-4f38-ad06-bc65fc8f0c7a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.157631 Oct 16 01:01:04.995677 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 245/459] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:04 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:05.317787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:05.318395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:05.325955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:05.326454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:05.327092 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:05.327877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:05.328495 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:05.329060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:05.343956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 HTTP/1.1 Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:01:05.345908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:01:05.347503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'f0dbf503-273a-4834-af7e-20ee0ac8b573'} Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:01:05.348428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:01:05.349818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:05.351119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:01:05.351119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ec06e337-151a-43e3-ae2b-0689f0d2cf73 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 404 len: 111 microversion: 2.1 time: 0.033749 Oct 16 01:01:05.351119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 215/460] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:05 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:01:05.357999 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-84f48c64-1dd0-4414-ac33-323c4a904e1b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '.version_select at 0x7f8bbda14c10>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:05.379394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-84f48c64-1dd0-4414-ac33-323c4a904e1b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1791417281" status: 202 len: 0 microversion: 2.1 time: 0.025395 Oct 16 01:01:05.379980 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 246/461] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:01:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1791417281 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:01:06.017728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:06.018271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:06.025561 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:06.026042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:06.026473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:06.027204 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:06.027655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:06.028073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:06.298462 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:06.307973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-87b67d12-8359-4df1-88c2-c0e7e24d9d87 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.294161 Oct 16 01:01:06.308655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 216/462] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:06 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:06.389546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b429b0dc-9d94-493f-9e80-c135bab2b3ed tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: .version_select at 0x7f8bbda6ba60>, body: {"keypair": {"name": "tempest-keypair-1171493286", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOKumgFI+seBy1Oid+zE6kqF1OSq6CsWKG6mTVb8z26oRpE9PiSYe6zGxvQVmKB+4fK79hFWmvPGM0w5AayvT5J2pHiC/y71N2TuZ3i3ZhFJJS6TKqzpqDdgGdn/tauIPw=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:06.394778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:INSERT=3,nova_api:DELETE=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:06.409470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b429b0dc-9d94-493f-9e80-c135bab2b3ed tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:06.413821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b429b0dc-9d94-493f-9e80-c135bab2b3ed tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:06.437106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b429b0dc-9d94-493f-9e80-c135bab2b3ed tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.052676 Oct 16 01:01:06.437793 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 247/463] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:01:06 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:07.331508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:07.334282 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:07.341241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:07.341729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:07.342528 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:07.343120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:07.343580 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:07.344226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:07.362296 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=100 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:07.508040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:07.516733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a9b7a674-ea84-4537-b910-ebb9402595b7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.189056 Oct 16 01:01:07.517750 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 217/464] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:07 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:07.815395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "name": "network-vif-plugged", "status": "completed", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:07.828099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:07.828704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:07.829258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:07.844218 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1644756918", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7dae3d76-90a2-41d2-9dad-7f7856d5ab85"}], "security_groups": [{"name": "tempest-securitygroup--1615976990"}], "key_name": "tempest-keypair-1171493286", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:07.864287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] Creating event network-vif-plugged:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:01:07.878217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-efe479dd-9714-489a-b494-b864f99c3fe7 req-55cfe356-f39e-453a-ac18-9edba2e006d4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067771 Oct 16 01:01:07.878952 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 248/465] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:01:07.971828 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] validate_networks() for [('7dae3d76-90a2-41d2-9dad-7f7856d5ab85', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:01:08.438957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:01:08.439556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:01:08.439986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:01:08.440432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:01:08.440844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:01:08.450332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:08.450818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:08.451362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:08.452087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:08.452541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:08.453020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:08.470344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:08.474504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:08.485971 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:01:08.488326 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:01:08.538699 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-32d382cf-2452-4334-a2bc-228e73d4eebd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.700372 Oct 16 01:01:08.539547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 218/466] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:01:07 2022] POST /compute/v2.1/servers => generated 402 bytes in 702 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:01:08.542208 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:08.543190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:08.550068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:08.550703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0ee881b-e377-4cd6-bd9f-4583bc7c52ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:08.550899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:08.551290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:08.551512 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0ee881b-e377-4cd6-bd9f-4583bc7c52ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:08.552308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:08.552951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:08.553531 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:08.627508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0ee881b-e377-4cd6-bd9f-4583bc7c52ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:08.637561 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0ee881b-e377-4cd6-bd9f-4583bc7c52ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1242 microversion: 2.1 time: 0.090728 Oct 16 01:01:08.638334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 219/467] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:08 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1242 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:08.764645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:08.773228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-306d69d0-2d7e-4c1c-99a0-247993c0c6ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1259 microversion: 2.1 time: 0.237302 Oct 16 01:01:08.773802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 249/468] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:08 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1259 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:09.210272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4ed0c81b-572c-4572-b561-c910c5cfac57"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:09.214586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:09.221520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.222106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.222725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:09.223764 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.224383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.225101 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:09.667078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:09.667646 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:09.677261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.679100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.680120 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:09.681037 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.681688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.682320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:09.787980 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:09.796401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bbfbefd2-5eaa-4b53-aced-79e8e25d20f9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1294 microversion: 2.1 time: 0.133404 Oct 16 01:01:09.797118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 250/469] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:09 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1294 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:09.801819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:09.802653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:09.815302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.816965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.817625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:09.819433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:09.821002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:09.821654 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:10.075656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b33f41ac-993f-4a43-a278-a1a76bc81a3e tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.870083 Oct 16 01:01:10.078490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 220/470] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:01:09 2022] POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments => generated 194 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:10.129040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:10.144021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-74562939-e55d-40f9-870f-1f0745e267c2 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1443 microversion: 2.1 time: 0.345539 Oct 16 01:01:10.152934 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 251/471] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:09 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1443 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:10.180118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:10.180891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:10.192991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:10.193557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:10.196036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:10.198356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:10.199407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:10.202186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:10.379666 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:10.389267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2fed86dc-1871-48ee-92ce-1ff749284fe9 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1443 microversion: 2.1 time: 0.214697 Oct 16 01:01:10.396877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 221/472] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:10 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1443 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:10.821283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:10.821891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:10.828416 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:10.828895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:10.829323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:10.830023 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:10.830462 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:10.830908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:10.945286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:10.956276 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-38ff7951-5c1c-45c5-ae0a-9efd1c11888f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1300 microversion: 2.1 time: 0.139621 Oct 16 01:01:10.956865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 252/473] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:10 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1300 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:11.982321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:11.983633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:11.991035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:11.991651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:11.993344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:11.994124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:11.994814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:11.995375 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:12.193033 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:12.205920 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-311e4568-7a66-4694-b23b-82e8dd295086 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.229385 Oct 16 01:01:12.207280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 222/474] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:11 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:12.538667 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c861342-ab71-4ab6-bed1-41f3b6e38853", "tag": "35e81c79-fcfc-45cc-82e6-d74a12d99947"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:12.564015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:12.565396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:12.570784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:12.602986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] Creating event network-changed:35e81c79-fcfc-45cc-82e6-d74a12d99947 for instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 on hv-cinder-maste Oct 16 01:01:12.608828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-f042d462-e5e7-4c95-a222-af0e335ac0de req-5c50b2bb-5b05-4692-b8f7-95001bf5423a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074934 Oct 16 01:01:12.616390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 253/475] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:13.245398 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:13.246844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:13.269830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:13.269830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:13.269830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:13.269830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:13.273697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:13.273697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:13.474299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:13.493553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3a625425-3664-43cb-be17-d7bf4130ea48 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.265690 Oct 16 01:01:13.494927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 223/476] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:13 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:14.349997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:14.365361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:14.366244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:14.367031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:14.381631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] Creating event network-changed:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:01:14.388416 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-10830db1-2a89-4a06-ae69-67f724b3f29a req-8b68afbf-0933-486b-b6be-66d53f3972e6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051403 Oct 16 01:01:14.389754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 254/477] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:14.518765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:14.519926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:14.535105 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:14.536577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:14.537263 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:14.543884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:14.548319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:14.550096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:14.937335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:14.954987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4dd867fd-c718-4032-8a24-0bbc0452db3e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.439909 Oct 16 01:01:14.955958 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 224/478] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:14 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:15.981502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:15.982375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:15.998900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:16.000290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:16.001375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:16.004084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:16.005147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:16.006751 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:16.279197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:16.292305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4e690a49-346a-4e0d-a0e3-bbc2cac71992 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.317059 Oct 16 01:01:16.292899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 255/479] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:15 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:17.315956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:17.317199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:17.330246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:17.331371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:17.333001 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:17.336977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:17.337673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:17.338267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:17.499366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:17.509477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a6a766b2-0334-41fe-8acf-e2e6903e7152 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.197536 Oct 16 01:01:17.510328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 225/480] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:17 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:18.020924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:18.023410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:18.029103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:18.032176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.032688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.033112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.033890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.034347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.034771 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.062561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lazy-loading 'flavor' on Instance uuid a14ab912-59d2-449f-b756-d19cad8d7b96 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:01:18.069309 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c861342-ab71-4ab6-bed1-41f3b6e38853", "name": "network-vif-plugged", "status": "completed", "tag": "35e81c79-fcfc-45cc-82e6-d74a12d99947"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:18.088533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.089264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.090571 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.117223 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] Creating event network-vif-plugged:35e81c79-fcfc-45cc-82e6-d74a12d99947 for instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 on hv-cinder-maste Oct 16 01:01:18.132518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-bd8a58a3-ecc2-40c0-894f-9ff49ca3b18a req-106320b6-df7f-44b0-b95b-9bae04a48723 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066581 Oct 16 01:01:18.133677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 226/481] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:18.247452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:18.276340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a5d95a9b-39d4-42b7-b5f9-09f6c56662e9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/action" status: 200 len: 32 microversion: 2.1 time: 0.259390 Oct 16 01:01:18.300482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:18.301146 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:18.304215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 256/482] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:01:18 2022] POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/action => generated 32 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:01:18.311851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.312374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.312816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.313546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.314090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.314536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.461622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:18.534891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:18.538388 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:18.545952 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.546609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.547337 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.549389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:18.550213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:18.550830 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:18.633390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=8 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:18.681780 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:18.696611 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c8175f2e-e9bb-4221-afa8-16d62730e4a1 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.411645 Oct 16 01:01:18.697231 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 227/483] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:18 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:18.829629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:18.839487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8a6af53e-e866-4294-b82e-8c28508345ae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.310491 Oct 16 01:01:18.840845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 257/484] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:18 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:19.723735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:19.724352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:19.732160 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:19.732667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:19.733107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:19.734023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:19.734482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:19.735515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:19.867045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:19.867917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:19.875679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:19.875870 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:19.876731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:19.876897 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:19.877302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:19.877722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:19.958790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:19.974156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-88d0a55e-a3ea-4004-88b4-f9fa53c1c85b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.251203 Oct 16 01:01:19.987343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 228/485] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:19 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:20.080573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:20.088546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-07c4b939-00e1-4aaf-8b5e-df7cb26bb601 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1368 microversion: 2.1 time: 0.228871 Oct 16 01:01:20.089447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 258/486] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:19 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1368 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:21.002310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:21.003896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:21.012745 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.013504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.014113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.015173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.015959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.016559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.117359 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:21.118229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:21.147240 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.148414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.149243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.150378 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.152215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.153390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.211408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:21.220961 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a8a40a4a-d3fd-44b6-a9a2-13e39205d4f8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.222797 Oct 16 01:01:21.221547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 229/487] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:20 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:21.346346 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:21.357476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4993ebc0-a1af-4abe-b1d3-f52261a51314 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1558 microversion: 2.1 time: 0.248319 Oct 16 01:01:21.370896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 259/488] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:21 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1558 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:21.381835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:21.382334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:21.388306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.388795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.389229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.389931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:21.390389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:21.390919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:21.482529 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-899b7fe9-5195-4637-88e1-fd4d7c76a732 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.104741 Oct 16 01:01:21.483892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 230/489] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:01:21 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:22.245563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:22.246185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:22.263169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:22.263832 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:22.264355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:22.265155 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:22.267631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:22.268178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:22.481521 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:22.498959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-376378d0-0763-4f76-82aa-3e3a27e70ca6 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.258044 Oct 16 01:01:22.500254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 260/490] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:22 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:23.215952 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0c861342-ab71-4ab6-bed1-41f3b6e38853", "tag": "35e81c79-fcfc-45cc-82e6-d74a12d99947"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:23.237817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:23.245114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:23.245833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:23.270118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] Creating event network-changed:35e81c79-fcfc-45cc-82e6-d74a12d99947 for instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 on hv-cinder-maste Oct 16 01:01:23.283520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-1b105ef8-5630-4111-8cbf-e10c044bdf99 req-32d00f64-e445-4cbf-bfa6-df2ceea81ce3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068832 Oct 16 01:01:23.284839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 231/491] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:23.528419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:23.529634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:23.543573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:23.546770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:23.550017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:23.551564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:23.552313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:23.552966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:23.769004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:23.778406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-348b4e5d-6395-46c0-aa6e-5ae373da010a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.253962 Oct 16 01:01:23.780250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 261/492] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:23 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:24.807646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:24.808339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:24.814787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:24.815321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:24.815760 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:24.816468 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:24.817128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:24.817725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:24.961886 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:24.971422 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-88566589-e44b-4fed-96cf-40c9e24c55de tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.168775 Oct 16 01:01:24.972004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 232/493] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:24 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:25.995684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:25.997254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:26.007735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:26.008850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:26.008850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:26.009806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:26.010355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:26.010902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:26.162718 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:26.172187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-22424b12-1415-4566-941a-8ea2a113d19f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.180969 Oct 16 01:01:26.172956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 262/494] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:25 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:27.198643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:27.200275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:27.207450 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:27.208281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:27.208860 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:27.210184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:27.212054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:27.212508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:27.404567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:27.416691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d05bf6b0-8a45-41ef-9fb8-022cf49f99e4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.222731 Oct 16 01:01:27.419179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 233/495] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:27 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:28.441502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:28.442180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:28.449527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:28.450065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:28.450529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:28.451255 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:28.452099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:28.452613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:28.607341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:28.617192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f07a7ef2-7170-47be-8ee8-b3d528ac0339 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.179425 Oct 16 01:01:28.618188 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 263/496] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:28 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:29.641437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:29.644359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:29.651992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:29.653688 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:29.654645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:29.655328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:29.656266 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:29.656942 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:29.657595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:29.812292 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:29.822883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-873e2ad8-a3a2-42c2-b1ea-605c57daaf0d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.185190 Oct 16 01:01:29.823474 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 234/497] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:29 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:30.846843 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:30.847487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:30.856709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:30.857371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:30.857813 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:30.858520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:30.859018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:30.859452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:31.015951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:31.025290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9cfac5dc-7222-4e34-bcb3-5bc5303a1d5d tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.182244 Oct 16 01:01:31.025902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 264/498] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:30 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:31.378506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7b4be1b4-00df-4a9c-8d22-f833c788e41e tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'create', calling method: .version_select at 0x7f8bbdd134c0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-890825791", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB0ExD3f/kzWd7chVZnDiSuinMoLCYEnuDHqO0dGn/s0l8/4yMmO8p88lGfRZhcF17/lubL7AN+apsrE4WI0is4MBx4/gDRa2norKwgwt2l+2w7fiVPH203wVFdkIoswBw=="}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:31.394271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-7b4be1b4-00df-4a9c-8d22-f833c788e41e tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Getting quotas for project 343c019304314c669a3381cc4c68e481. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:31.398653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-7b4be1b4-00df-4a9c-8d22-f833c788e41e tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Getting quotas for user a49486f43ff942bcbf4ef782d6eb6238 and project 343c019304314c669a3381cc4c68e481. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:31.427955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7b4be1b4-00df-4a9c-8d22-f833c788e41e tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.049263 Oct 16 01:01:31.427955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 235/499] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:01:31 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:31.476671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1577023133", "imageRef": "67c57bbb-c541-4b1c-8944-5efb5692a2f7", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-890825791", "networks": [{"uuid": "3e12ae87-77ee-4d05-8dce-be1b6f55d51c"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:31.533458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] validate_networks() for [('3e12ae87-77ee-4d05-8dce-be1b6f55d51c', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:01:31.913685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:01:31.915147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:01:31.915932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:01:31.916541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:01:31.917192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:01:31.926271 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:31.926757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:31.927238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:31.928465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:31.928602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:31.928920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:31.948719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Getting quotas for project 343c019304314c669a3381cc4c68e481. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:31.953717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Getting quotas for user a49486f43ff942bcbf4ef782d6eb6238 and project 343c019304314c669a3381cc4c68e481. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:31.965586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:01:31.967880 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='67c57bbb-c541-4b1c-8944-5efb5692a2f7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:01:32.014084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-65015493-c884-4148-b488-af0f7ed6c784 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.541838 Oct 16 01:01:32.014732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 265/500] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:01:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 543 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:01:32.026359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-11beff04-1ac3-4487-932e-4e149b7ac7cb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:32.027113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-11beff04-1ac3-4487-932e-4e149b7ac7cb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:32.049446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:32.050139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:32.057204 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:32.057772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:32.058288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:32.059094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:32.059623 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:32.060148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:32.082276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-11beff04-1ac3-4487-932e-4e149b7ac7cb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:32.091187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-11beff04-1ac3-4487-932e-4e149b7ac7cb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1258 microversion: 2.1 time: 0.069215 Oct 16 01:01:32.091877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 236/501] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:32 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1258 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:32.201031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:32.209197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a0789b6b-2b49-4448-b60e-32b2c526fbf5 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.163699 Oct 16 01:01:32.209978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 266/502] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:32 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:33.114106 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:33.114722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:33.121863 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:33.122350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:33.122791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:33.123517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:33.123974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:33.124403 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:33.186094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:33.194166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-30470696-2634-4608-9100-677bbaeae57a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1310 microversion: 2.1 time: 0.083876 Oct 16 01:01:33.195257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 237/503] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:33 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1310 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:33.233619 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:33.234244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:33.240747 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:33.241275 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:33.242041 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:33.242765 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:33.243259 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:33.243689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:33.388066 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:33.396704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c674e9c7-1922-4d02-b7f5-88f5e858026f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.167014 Oct 16 01:01:33.397917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 267/504] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:33 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:34.220383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:34.221165 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:34.228288 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:34.228864 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:34.229362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:34.230165 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:34.230677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:34.231122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:34.234003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:34.377504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:34.388235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1a4d932f-8e74-4a61-bb57-e667fc1dcb41 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.172794 Oct 16 01:01:34.388875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 238/505] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:34 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:34.421071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:34.421721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:34.429073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:34.429694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:34.430450 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:34.431045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:34.431507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:34.431937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:34.575268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:34.583728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-26fabd43-374f-4703-ab35-a560f151c4b0 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.166610 Oct 16 01:01:34.584300 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 268/506] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:34 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:35.414125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:35.415207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:35.423341 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:35.424139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:35.424582 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:35.425299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:35.425763 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:35.426278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:35.573123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:35.583036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-00dbcc68-87cc-46b4-a2c3-3ff8f1d22eca tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.174059 Oct 16 01:01:35.583629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 239/507] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:35 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:35.608170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:35.608891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:35.615463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:35.616608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:35.617348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:35.618143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:35.618604 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:35.619245 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:35.757945 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:35.766289 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-646621d4-5736-46c5-9998-308416a699cf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.162061 Oct 16 01:01:35.766784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 269/508] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:35 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:36.605799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:36.606406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:36.612981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:36.613473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:36.613920 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:36.614628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:36.615113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:36.615551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:36.753018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:36.761852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ce91e6fb-c2d0-46dc-8fff-6e4f67279b8d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.159843 Oct 16 01:01:36.777796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 240/509] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:36 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:36.789902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:36.791156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:36.793600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:36.799247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:36.799736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:36.800402 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:36.801107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:36.801568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:36.801997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:36.957034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:36.965758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d3290830-eb7a-479b-acca-012355df7d61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.180026 Oct 16 01:01:36.966357 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 270/510] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:36 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:37.785316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:37.786075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:37.793048 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:37.793542 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:37.793980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:37.794682 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:37.795174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:37.795603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:37.929054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:37.939252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-580cea45-209b-4ab7-9ed5-ed8b13293825 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.157529 Oct 16 01:01:37.940321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 241/511] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:37 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:37.989662 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:37.990347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:37.997853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:37.998352 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:37.998795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:37.999539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:38.000003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:38.000441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:38.138061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:38.146764 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6f2f5a73-09e1-4ffa-94fb-631b984155bd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.161005 Oct 16 01:01:38.158171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 271/512] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:37 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:38.963845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:38.964563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:38.971281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:38.971891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:38.972413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:38.973197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:38.973728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:38.974218 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:39.116961 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:39.125468 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e9ce51ce-5c11-40af-99a9-8bf9a8c763a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.166191 Oct 16 01:01:39.126400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 242/513] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:38 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:39.170392 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:39.171785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:39.179103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:39.179689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:39.180221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:39.180937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:39.181397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:39.181825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:39.321380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:39.335019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4d110c01-39df-496d-b8b1-6a7f88e6c6bf tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.168536 Oct 16 01:01:39.336613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 272/514] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:39 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:40.150498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:40.151179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:40.157728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:40.158251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:40.158636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:40.159367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:40.159820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:40.160240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:40.318043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:40.329282 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-70b73d63-9b8b-48b4-9ee4-7860c6e5b4e0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.183435 Oct 16 01:01:40.329873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 243/515] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:40 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:40.359964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:40.360545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:40.368314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:40.368812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:40.369257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:40.369970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:40.370464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:40.371031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:40.537517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:40.546691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:40.551612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-03f10cf1-5761-49c5-adbb-0ada50af0fc4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.195826 Oct 16 01:01:40.567043 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 273/516] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:40 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:41.357841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:41.358500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:41.364910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:41.365391 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:41.365824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:41.366520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:41.366995 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:41.367419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:41.517036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:41.532405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-49ddb2ac-47cd-49d2-be97-a6584bff672b tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.178146 Oct 16 01:01:41.535908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 244/517] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:41 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:41.589997 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:41.590695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:41.598544 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:41.599106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:41.599562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:41.600574 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:41.601081 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:41.601663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:41.736383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:41.746320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-37360c99-861e-47fc-841d-ed364535bda7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.161179 Oct 16 01:01:41.746933 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 274/518] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:41 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:42.558510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:42.560005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:42.563661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:42.567283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.568034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.568499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.569528 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.570362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.570834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.686584 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:42.687507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:42.691690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:42.693132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:42.701334 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.701828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.702266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.702993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.704320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.704926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.741701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a9679a47-84e6-40ce-8734-34f05e89f1e2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments/f3037e6a-d239-4f27-aea5-b4ee17a7b214" status: 200 len: 194 microversion: 2.20 time: 0.063952 Oct 16 01:01:42.742526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 275/519] 10.222.0.25 () {60 vars in 1454 bytes} [Sun Oct 16 01:01:42 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments/f3037e6a-d239-4f27-aea5-b4ee17a7b214 => generated 194 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 01:01:42.745859 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:42.756900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-71883310-03cb-4d75-9d73-df8d23dd0b8c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.202970 Oct 16 01:01:42.758521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 245/520] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:42 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:42.769758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:42.770362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:42.777188 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.777697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.778137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.778820 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.779390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.779762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.825557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:42.826055 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:42.832456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.833206 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.833988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.835625 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:42.836520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:42.837042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:42.950268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:42.959296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ca22df0d-2880-4f14-b62f-dced3c20e9a4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.193331 Oct 16 01:01:42.970916 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 276/521] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:42 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:42.988406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lazy-loading 'services' on Instance uuid d07b6833-a39f-462d-9c7e-48dda346d8cb {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:01:43.121652 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6106256f-fb82-4ceb-af33-6666fc7b19b0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments/f3037e6a-d239-4f27-aea5-b4ee17a7b214" status: 202 len: 0 microversion: 2.20 time: 0.301013 Oct 16 01:01:43.122268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 246/522] 10.222.0.25 () {60 vars in 1457 bytes} [Sun Oct 16 01:01:42 2022] DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments/f3037e6a-d239-4f27-aea5-b4ee17a7b214 => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:01:43.132921 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:43.135782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:43.142956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.143476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.144143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:43.144928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.145600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.146261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:43.181517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-69167d7c-1452-4560-9d4a-f968053322e5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054109 Oct 16 01:01:43.182440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 277/523] 10.222.0.25 () {60 vars in 1343 bytes} [Sun Oct 16 01:01:43 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 01:01:43.783863 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:43.784558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:43.791621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.792201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.792645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:43.793343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.793827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.794256 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:43.925691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:43.934303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-388f1d1d-82ba-4fd6-b0c7-802e0c464928 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.155254 Oct 16 01:01:43.934914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 247/524] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:43 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:43.985566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:43.986139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:43.993469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.993955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.994418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:43.995162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:43.995657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:43.996112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.127982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:44.136283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-43af1c2a-3293-429a-9f50-748585a3a3c8 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.154692 Oct 16 01:01:44.136847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 278/525] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:43 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:44.195184 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:44.197442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:44.203754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:44.204240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:44.204678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.205374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:44.205832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:44.206261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.240995 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9b340755-a85d-4374-b3db-048c2c07cc47 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049834 Oct 16 01:01:44.241518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 248/526] 10.222.0.25 () {60 vars in 1343 bytes} [Sun Oct 16 01:01:44 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 01:01:44.945711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:44.956061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:44.956681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:44.957425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.958385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:44.959026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:44.966446 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:44.967504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:44.968140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.969587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:44.970408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:44.971217 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:44.973186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] Creating event network-changed:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:01:44.980407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-13d30761-21e7-4afd-a69d-c4e5c62cdd61 req-b9cf71ff-7e6b-4165-8090-0b5a0188db06 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039037 Oct 16 01:01:44.981505 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 279/527] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:45.112590 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:45.120796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-955b867d-bf66-4738-9e2b-5e5d75047446 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.166786 Oct 16 01:01:45.121366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 249/528] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:44 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:45.161340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:45.162034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:45.168512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:45.168930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:45.169368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:45.170059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:45.170510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:45.170962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:45.253904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:45.256226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:45.262502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:45.263031 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:45.263471 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:45.264175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:45.264641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:45.265069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:45.296107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d50a143c-7efd-4683-9632-bc7c839786c2 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046549 Oct 16 01:01:45.297040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 250/529] 10.222.0.25 () {60 vars in 1343 bytes} [Sun Oct 16 01:01:45 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 01:01:45.309052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:45.317389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-270a6cf0-8a3d-4a43-85ec-44871e172ff7 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.160778 Oct 16 01:01:45.317990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 280/530] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:45 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:46.144168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:46.144814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:46.151623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.152110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.152547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.153251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.153702 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.154128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.308544 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:46.311058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:46.318262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.319419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.319937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.320661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.321121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.321548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.356541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8386094b-66b0-4a0c-85d6-0dd394a5d74a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.052158 Oct 16 01:01:46.357263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 281/531] 10.222.0.25 () {60 vars in 1343 bytes} [Sun Oct 16 01:01:46 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:46.361992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:46.362692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:46.372709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.373217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.373676 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.375083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.375563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.376006 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.510801 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:46.519575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-80cb32e1-5bbd-4603-8bf1-d75858760d1f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.161528 Oct 16 01:01:46.520242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 282/532] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:46 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:46.525686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:46.526455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:46.533094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.533802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.534244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.534966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.535425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.535853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.563194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:01:46.642965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e4cc8daf-9642-40fc-9d48-994a2c25048f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 204 len: 0 microversion: 2.20 time: 0.121198 Oct 16 01:01:46.643690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 283/533] 10.222.0.25 () {60 vars in 1280 bytes} [Sun Oct 16 01:01:46 2022] DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 16 01:01:46.654233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:46.654953 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:46.661033 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.661495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.661927 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.662613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:46.663651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:46.664260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:46.708437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:46.718755 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-269bc622-db6f-4e5b-b547-1a1ddc718a8f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.578367 Oct 16 01:01:46.719759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 251/534] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:46 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:46.798054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:46.806518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:46.808089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:46.809261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b75ba421-cae5-407f-b211-5216978b8f60 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.160692 Oct 16 01:01:46.809833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 284/535] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:46 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:01:47.543875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:47.544496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:47.550989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.551478 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.551922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.552616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.553083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.554203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.687361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:47.695707 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-648dbc71-6969-41cd-8dc3-a5827853aa8c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.155823 Oct 16 01:01:47.696289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 252/536] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:47 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:47.742548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:47.743201 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:47.749667 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.750150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.750588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.751319 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.751775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.752251 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.835783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:47.836425 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:47.842519 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.843279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.843712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.844406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:47.844856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:47.845274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:47.886856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:47.894933 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f53f3397-b09a-46b3-aedc-f5725a21d858 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.157043 Oct 16 01:01:47.895530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 285/537] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:47 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:47.986387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:47.995095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:47.996544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:47.997714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0f85be6e-7ed8-42fb-bc4b-52cae35def88 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.165952 Oct 16 01:01:47.998279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 253/538] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:47 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:01:48.719611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:48.720134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:48.726228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:48.726701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:48.727166 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:48.727845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:48.728292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:48.728717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:48.861152 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:48.870085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-59828bc5-7180-4a30-9e9d-b2578568c480 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1734 microversion: 2.1 time: 0.154414 Oct 16 01:01:48.870663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 286/539] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:48 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1734 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:48.918621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:48.919640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:48.930036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:48.930549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:48.931020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:48.931729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:48.932188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:48.932615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:48.937034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:49.023031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:49.023658 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:49.029568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:49.030042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:49.030525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:49.031213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:49.031659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:49.032077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:49.092800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:49.101339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-aec79a66-a1c7-4f6d-b44f-38a755a04ed7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.186635 Oct 16 01:01:49.102155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 254/540] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:48 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:49.163331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:49.171527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:49.172996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:49.174172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e8f0fb00-5465-4b2f-a1ad-262dac3c7dba tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1721 microversion: 2.20 time: 0.155085 Oct 16 01:01:49.174729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 287/541] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:49 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1721 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:01:49.893693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:49.894479 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:49.901554 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:49.902127 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:49.902551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:49.903325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:49.903782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:49.904329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.124905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:50.125539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:50.132857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.133286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.133712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.134667 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.134800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.135920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.136431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.144083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-57f6b37d-024f-4c7a-8d96-a623e1cb5e6b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1714 microversion: 2.1 time: 0.254226 Oct 16 01:01:50.153018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 255/542] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:49 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1714 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:50.210515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:50.211655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:50.220437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.221045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.221605 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.222575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.223168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.223733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.286918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.295178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9c3cb196-8bae-4aea-bce7-bbf67a930595 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.174117 Oct 16 01:01:50.295757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 288/543] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:50 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:50.300423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:50.300919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:50.307784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.308288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.308734 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.309440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.309899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.310329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.310558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.328907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.329626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:50.330694 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.332671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-317f7c85-71c1-432e-910b-832d77525eef tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1654 microversion: 2.20 time: 0.132878 Oct 16 01:01:50.333423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 256/544] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:50 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1654 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:50.415078 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] HTTP exception thrown: Cannot 'detach_volume' instance a14ab912-59d2-449f-b756-d19cad8d7b96 while it is in vm_state rescued Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Request method failure captured: Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments/4ed0c81b-572c-4572-b561-c910c5cfac57 HTTP/1.1 Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b-xNm9KCVjgM3wqxu3_jLffliMpX0rEBJ2pjnEEcTaHDwNHHNLws3WGBBUFguF0p4v0Uh6PcQ0OWevJVzKcLOchCd-DjlwU0EBAQ0HWFl9B8ZLgM7suC6TrG-0QTAhRmCIJzo4n5DR8fd3oJDcM_OYmQ6Ak0HH1w5jq7S_PXbqbcA Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:01:50.416748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5462fcbfca114c8885f77f42b479b13d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5462fcbfca114c8885f77f42b479b13d"}]}] Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: b01ce2d27bdb4c91a3f55c891f949e00 Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:01:50.418403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Cannot 'detach_volume' instance a14ab912-59d2-449f-b756-d19cad8d7b96 while it is in vm_state rescued Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'server_id': 'a14ab912-59d2-449f-b756-d19cad8d7b96', 'id': '4ed0c81b-572c-4572-b561-c910c5cfac57'} Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:01:50.419301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance a14ab912-59d2-449f-b756-d19cad8d7b96 while it is in vm_state rescued Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance a14ab912-59d2-449f-b756-d19cad8d7b96 while it is in vm_state rescued Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:01:50.421042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Returning 409 to user: Cannot 'detach_volume' instance a14ab912-59d2-449f-b756-d19cad8d7b96 while it is in vm_state rescued {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:01:50.422358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ceb1ffa0-4138-4473-91e2-2f96b5132f61 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments/4ed0c81b-572c-4572-b561-c910c5cfac57" status: 409 len: 152 microversion: 2.1 time: 0.121479 Oct 16 01:01:50.422358 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 289/545] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:01:50 2022] DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments/4ed0c81b-572c-4572-b561-c910c5cfac57 => generated 152 bytes in 123 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 16 01:01:50.427143 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:50.427672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:50.433819 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.434308 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.434749 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.435496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.435944 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.436369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.505777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-53451367-9287-4c7d-bf51-9972dbf70524 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/action" status: 202 len: 0 microversion: 2.1 time: 0.082298 Oct 16 01:01:50.506934 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 257/546] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:01:50 2022] POST /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:01:50.515219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:50.515831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:50.521827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.522956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.523395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.524096 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.524552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.524981 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.624051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d07b6833-a39f-462d-9c7e-48dda346d8cb", "name": "network-vif-deleted", "tag": "9f15f027-a9c6-4298-a989-336905ee5a12"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:50.634477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:50.634984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:50.635417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:50.647045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] Creating event network-vif-deleted:9f15f027-a9c6-4298-a989-336905ee5a12 for instance d07b6833-a39f-462d-9c7e-48dda346d8cb on hv-cinder-maste Oct 16 01:01:50.652887 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-b79a3180-053d-4c8e-9e18-ea05d74fe346 req-2a38651f-de12-4a93-bc79-dce78bea3cf2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032867 Oct 16 01:01:50.654667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 258/547] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:01:50.672824 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:50.682239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-03472463-c4e6-4b12-9495-1f19657ded99 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1722 microversion: 2.1 time: 0.170780 Oct 16 01:01:50.683090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 290/548] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:50 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:51.319534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:51.320238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:51.326904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.327635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.328075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.328957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.329406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.329830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.365348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:51.366601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:51.373329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.373816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.374252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.374989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.375448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.375870 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.408281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.408761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.409192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.409633 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lazy-loading 'fault' on Instance uuid d07b6833-a39f-462d-9c7e-48dda346d8cb {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:01:51.455906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:51.466304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:51.467818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:51.469175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4ad7d421-b148-4dca-943e-13c152ba6ab3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 200 len: 1369 microversion: 2.20 time: 0.107816 Oct 16 01:01:51.478030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 291/549] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:51 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:01:51.500164 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:51.508997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2124c006-ef24-4b8f-9770-81d1e902847c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.194269 Oct 16 01:01:51.509602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 259/550] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:51 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:51.707550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:51.708258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:51.715246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.716040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.716659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.717373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:51.717836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:51.718272 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:51.848473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:51.856843 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d0629fe1-ca23-4770-858f-fc8e353ea3c4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1722 microversion: 2.1 time: 0.154041 Oct 16 01:01:51.857421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 292/551] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:51 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1722 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:52.494624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:52.495262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:52.501673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.502352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.502990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:52.503692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.504144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.504573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:52.519084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb HTTP/1.1 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:01:52.520735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:01:52.522959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'd07b6833-a39f-462d-9c7e-48dda346d8cb'} Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:01:52.523867 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:01:52.525330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:01:52.526615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:01:52.526615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-23ec652a-2255-457d-aeb3-39c5adb4f1b8 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 404 len: 111 microversion: 2.20 time: 0.032176 Oct 16 01:01:52.526615 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 260/552] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:52 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:01:52.532490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:52.532649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5add713d-a605-4da9-89f9-5dbbfcfefc30 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '.version_select at 0x7f8bbf2b2040>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:52.533082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:52.540020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.540524 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.540954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:52.541653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.542098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.542523 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:52.553626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5add713d-a605-4da9-89f9-5dbbfcfefc30 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-8122179" status: 204 len: 0 microversion: 2.20 time: 0.024974 Oct 16 01:01:52.554400 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 293/553] 10.222.0.25 () {60 vars in 1253 bytes} [Sun Oct 16 01:01:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-8122179 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 16 01:01:52.721994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:52.730623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ce099d18-8979-4acf-b494-7d859d3b0f87 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.201626 Oct 16 01:01:52.731235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 261/554] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:52 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:52.881438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:52.882269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:52.888669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.889083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.889522 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:52.890268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:52.890663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:52.891119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:53.129221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:53.137546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b8a84d87-e69e-4501-b931-700c79292833 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1722 microversion: 2.1 time: 0.260461 Oct 16 01:01:53.138124 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 294/555] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:52 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1722 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:53.361159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8cda71b-88d1-441a-b52a-85757b5728ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: .version_select at 0x7f8bbdbe3670>, body: {"keypair": {"name": "tempest-keypair-532904353", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC+zZZ4cmyDZhbwZdZzZQs9Y3lMQjMmioLL980/8U3PK1dru5b180zVUak8bIpqdvIzHDpMSvocGgZtjx5d3f7NMYyybqb7RAe3EVtQCJknZIJIfrzz68+OZvf/AECVzDA=="}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:53.376870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-c8cda71b-88d1-441a-b52a-85757b5728ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:53.380703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-c8cda71b-88d1-441a-b52a-85757b5728ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for user ccece85a85fb48c4887b52c96c5bc642 and project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'key_pairs'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:53.405276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c8cda71b-88d1-441a-b52a-85757b5728ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.048106 Oct 16 01:01:53.405881 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 262/556] 10.222.0.25 () {62 vars in 1200 bytes} [Sun Oct 16 01:01:53 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 49 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 16 01:01:53.753931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:53.754526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:53.761794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:53.762282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:53.762717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:53.763459 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:53.763910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:53.764383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:53.924407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:53.938546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1fc92934-5e2b-416c-b0ea-cf01ed74704f tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.188479 Oct 16 01:01:53.951197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 295/557] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:53 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:54.029767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "name": "network-vif-plugged", "status": "completed", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:54.043830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.044473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.045125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.057426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] Creating event network-vif-plugged:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:01:54.062855 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-0ff1f17a-eb53-4a92-afe3-fac295850b37 req-11da47a9-995e-4bc3-85d2-a59609f9eced service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037405 Oct 16 01:01:54.064170 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 263/558] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:54.161982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:54.163396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:54.172750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.173797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.174823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.175986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.176736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.177493 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.320098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:54.329354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-92a3531f-105e-4ee1-abf2-f9f74e88f08f tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1728 microversion: 2.1 time: 0.171710 Oct 16 01:01:54.330021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 296/559] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:54 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1728 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:54.353828 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:54.354403 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:54.360710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.361199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.361631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.362339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.362824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.363448 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.478989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4257e9f1-9148-4320-9580-254f62793afb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129776 Oct 16 01:01:54.479850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 264/560] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:01:54 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-interface => generated 284 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:54.870049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-924359101", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "758abdbd-543f-42fe-8aa7-53af6cbbeb5d"}], "security_groups": [{"name": "tempest-securitygroup--1814361043"}], "key_name": "tempest-keypair-532904353", "user_data": "}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:54.966131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:54.966738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:54.973030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.973508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.973940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:54.974637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:54.976024 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:54.976448 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:55.118459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:55.126389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-18212838-7a19-434e-a591-6ab499196bdb tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1358 microversion: 2.1 time: 0.164244 Oct 16 01:01:55.127020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 265/561] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:54 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1358 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:55.278134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] validate_networks() for [('758abdbd-543f-42fe-8aa7-53af6cbbeb5d', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:01:55.931338 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:55.941059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:55.941803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:55.942411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:55.957252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] Creating event network-changed:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:01:55.963441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-6c1d277b-9b6a-4a98-bae2-0ca3c103cf30 req-ffb7e1e6-34c3-4d05-bdc7-02aeb3c4c8e2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035616 Oct 16 01:01:55.964065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 266/562] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:01:55.978036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:01:55.978036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:01:55.978036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:01:55.978036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:01:55.978036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:01:55.989900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:55.990722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:55.991263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:55.992617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:55.993091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:55.993545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.011781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:01:56.016242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Getting quotas for user ccece85a85fb48c4887b52c96c5bc642 and project cca355ef4a294ecd88cc299b7e0ca9d8. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:01:56.029689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:01:56.032138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:01:56.091461 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8931d706-ff82-406a-a320-27a400c5f547 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.225817 Oct 16 01:01:56.098402 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 297/563] 10.222.0.25 () {62 vars in 1188 bytes} [Sun Oct 16 01:01:54 2022] POST /compute/v2.1/servers => generated 402 bytes in 1228 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 16 01:01:56.104316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:56.105090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:56.149586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:56.150341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:56.159360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.160212 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.160908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.161825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.162415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.163116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.202382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.215648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.217544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.221457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-859b4b47-cb25-471e-989d-d7b8c6c9b6d7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1280 microversion: 2.20 time: 0.120284 Oct 16 01:01:56.223768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 267/564] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:56 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1280 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:56.358071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.369118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-77cdb7bf-67f8-44ea-b1ef-5bf1a938a678 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1546 microversion: 2.1 time: 0.223377 Oct 16 01:01:56.370530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 298/565] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:56 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1546 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:01:56.395143 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:56.396730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:56.407114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.408491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.409299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.410597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.411390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.412124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.629616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.638800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5bead80f-78a6-4e41-b5b9-e19fc22fb9a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1546 microversion: 2.1 time: 0.249725 Oct 16 01:01:56.639420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 268/566] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:56 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1546 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:56.662638 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:56.672159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:56.672985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Searching by: {'deleted': False, 'project_id': '343c019304314c669a3381cc4c68e481'} {{(pid=119012) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 16 01:01:56.681752 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.682360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.682885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.683814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.684891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.685337 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.785814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.multi_cell_list [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=119012) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 16 01:01:56.891874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5575280c-72d3-4e33-b911-65f189764fa0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.233364 Oct 16 01:01:56.892484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 299/567] 10.222.0.25 () {58 vars in 1125 bytes} [Sun Oct 16 01:01:56 2022] GET /compute/v2.1/servers => generated 343 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:56.908709 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:56.910324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:56.918384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.919190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.919852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:56.920792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:56.921466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:56.922374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:57.071565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:57.082094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b45aacd2-6e55-4acb-b3c6-281ffb2dee92 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1546 microversion: 2.1 time: 0.177763 Oct 16 01:01:57.083195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 269/568] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:01:56 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1546 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:01:57.255663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:57.256855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:57.264718 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:57.265279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:57.265778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:57.266724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:57.267277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:57.267719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:57.332737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:57.343399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:57.344855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:57.345969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3f9e7feb-e732-4be7-a7e6-aa01a3466ced tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1332 microversion: 2.20 time: 0.094617 Oct 16 01:01:57.351043 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 300/569] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:57 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1332 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:57.980093 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:57.990924 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:57.991410 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:57.991839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:58.003668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] Creating event network-changed:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:01:58.008693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-803380ce-15d4-44d6-b7e2-be5a15577309 req-3646b152-2db8-4d28-9aaf-31c464fbeda0 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032437 Oct 16 01:01:58.009413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 270/570] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:58.371713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:58.372303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:58.378687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:58.379223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:58.379657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:58.380354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:58.380812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:58.381243 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:58.468361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:58.478237 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:58.479708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:58.481234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-58d99945-cb07-433d-a04a-376502b8ede4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1338 microversion: 2.20 time: 0.113607 Oct 16 01:01:58.481594 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 301/571] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:58 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1338 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:58.678534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "992410fd-c57f-46d6-8158-3244996d4182"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:58.680960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:58.687032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:58.687513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:58.687941 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:58.688634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:58.689712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:58.690148 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:58.979540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b902e849-9d78-4adc-a2aa-d1fef62b8bb3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.305056 Oct 16 01:01:58.980163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 271/572] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:01:58 2022] POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 194 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:01:59.508134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:01:59.509608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:01:59.516348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:59.516831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:59.517266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:59.518695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:59.522101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:59.522580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:59.643350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:01:59.650481 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:01:59.660663 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:01:59.661890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:01:59.663818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:01:59.666910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:59.676004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] Creating event network-changed:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:01:59.677753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:59.680113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:01:59.681505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-146082d6-991b-41eb-a57b-8a8b54e85d50 req-7604e144-0742-4e08-9730-4ab209bc0a43 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044861 Oct 16 01:01:59.681895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ab4533de-e2c9-43f5-b307-d45041e4a7a1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.177433 Oct 16 01:01:59.682767 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 302/574] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:01:59 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:01:59.686476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 272/574] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:01:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:00.708405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:00.709468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:00.716788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:00.717264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:00.717706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:00.718386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:00.718837 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:00.719283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:00.847477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:00.856874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:00.858306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:00.859466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8e438d54-5ce3-4f47-b6c6-e2a5a9b6d7ae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.155209 Oct 16 01:02:00.860030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 303/575] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:00 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:01.884642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:01.885339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:01.891601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:01.892087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:01.892549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:01.893324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:01.893809 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:01.894284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:02.041608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:02.049804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:02.051277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:02.052400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f49e1ed3-0f5d-48a8-8047-6740e251ed8c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.172674 Oct 16 01:02:02.052970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 273/576] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:01 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:02:03.079273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:03.079915 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:03.086353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:03.086890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:03.087352 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:03.088058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:03.088511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:03.088935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:03.221298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:03.229901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:03.231376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:03.232481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ec9c36ee-294d-41a8-99e4-cd132cb6c9aa tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.157151 Oct 16 01:02:03.233050 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 304/577] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:03 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:04.018318 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "name": "network-vif-plugged", "status": "completed", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:04.030147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:04.030630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:04.031095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:04.042784 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] Creating event network-vif-plugged:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:02:04.048416 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-e27eb1b4-43c7-41a1-b4e5-bc738b518569 req-13337298-01cc-44ab-9160-0b714fa3c33d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033218 Oct 16 01:02:04.048614 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 274/578] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:02:04.257563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:04.258669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:04.265373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:04.265862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:04.266304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:04.267049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:04.267507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:04.267934 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:04.404659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:04.412899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:04.414428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:04.416067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b8a47455-63bc-48cc-aa44-c154fcd0ed6b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.162372 Oct 16 01:02:04.416689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 305/579] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:04 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:05.442008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:05.442706 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:05.449316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:05.449814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:05.450256 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:05.451000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:05.451467 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:05.451898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:05.466889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:05.593036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:05.600992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:05.602969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:05.604063 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-056bfc0b-89c8-4e40-bd7a-4c5ab7dc3ff0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1406 microversion: 2.20 time: 0.166881 Oct 16 01:02:05.604621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 275/580] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:05 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1406 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:02:06.602852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:06.603490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:06.605816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:06.611489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.611978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.612417 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.613117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.613576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.614005 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.629236 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:06.629841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:06.631898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:06.638457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.638966 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.639402 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.640103 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.640553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.640973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.738583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:06.753211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:06.762282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-eef6be8f-41d5-4b18-a7ca-d032871b2884 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1592 microversion: 2.1 time: 0.163346 Oct 16 01:02:06.762876 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 306/581] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:06 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1592 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:06.773484 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:06.782017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:06.783493 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:06.784702 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-df4705ef-4aba-4dc8-a9c7-6dcdff020899 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1597 microversion: 2.20 time: 0.159396 Oct 16 01:02:06.785558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 276/582] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:06 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1597 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:02:06.810588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:06.811157 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:06.813244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:06.818754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.819348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.819788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.821103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:06.821562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:06.822097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:06.827795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:06.905403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0ce9a7e8-ae01-418e-a86d-68235d2c002b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.098740 Oct 16 01:02:06.906016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 307/583] 10.222.0.25 () {60 vars in 1316 bytes} [Sun Oct 16 01:02:06 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 16 01:02:07.885313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:07.895979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:07.896466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:07.896902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:07.908968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] Creating event network-changed:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:02:07.914097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-a6bd6d65-d365-45d6-8376-8059e3e3d61e req-35a06945-e4b8-4702-a1f3-825e34d8e209 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032830 Oct 16 01:02:07.918278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 277/584] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:08.851832 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:08.852688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:08.859842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:08.860402 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:08.860890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:08.861726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:08.863347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:08.864061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:08.896708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-93013101-725d-402f-956e-ff968f912f3c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/ips" status: 200 len: 149 microversion: 2.1 time: 0.049497 Oct 16 01:02:08.898015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 308/585] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:02:08 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/ips => generated 149 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:09.150701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:09.176137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:09.177977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:09.178709 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:09.196349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] Creating event network-changed:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:02:09.204980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-907e471a-290b-4ad8-8e2e-e4347d28c1e7 req-b296d774-e018-404c-8e12-4cc727f25513 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058751 Oct 16 01:02:09.206353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 278/586] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:10.263895 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-932958323"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:10.265102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:10.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:10.272221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:10.272660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:10.273364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:10.274431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:10.275185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:10.361839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d190657e-cba0-4b52-bdd4-149114e9c202"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:10.364389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:10.371671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:10.372320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:10.372752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:10.373463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:10.373900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:10.374321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:10.417159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.network.security_group_api [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Adding security group ef62311a-4a30-42aa-bd68-cc2deadeb13a to port c570709d-29cc-4c4f-a39b-9bfd4da608ed Oct 16 01:02:10.645426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-40140029-8fc0-4bfa-ac7f-dc3c4cb07502 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.288062 Oct 16 01:02:10.646775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 279/587] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:02:10 2022] POST /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments => generated 194 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:02:11.022593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ebf8c033-c264-4a2c-873b-74cf5e9b8f80 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action" status: 202 len: 0 microversion: 2.1 time: 0.762895 Oct 16 01:02:11.023253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 309/588] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:02:10 2022] POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action => generated 0 bytes in 765 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:02:11.036446 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:11.037155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:11.042671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:11.052878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:11.053376 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:11.053815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:11.054525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:11.055736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:11.056166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:11.056762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:11.057241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:11.057668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:11.077150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] Creating event network-changed:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:02:11.086046 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-1ffe6e8e-c9ae-4aae-a4c5-62d1cba03335 req-ad35811f-fbd9-4e13-8b6d-b327d6b9b2d5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052279 Oct 16 01:02:11.087628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 310/589] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:11.297682 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:11.308955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c27b6023-866c-4f5a-962b-06a87ae9b5c6 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1757 microversion: 2.1 time: 0.280214 Oct 16 01:02:11.310406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 280/590] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:11 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1757 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:18.194635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d190657e-cba0-4b52-bdd4-149114e9c202"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:18.198076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:18.205709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.206196 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.206634 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.207364 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.207823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.208253 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.458137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: POST /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments HTTP/1.1 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 74 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:02:18.459932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {"volumeAttachment": {"volumeId": "d190657e-cba0-4b52-bdd4-149114e9c202"}} Oct 16 01:02:18.462463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'server_id': '0c861342-ab71-4ab6-bed1-41f3b6e38853', 'body': {'volumeAttachment': {'volumeId': 'd190657e-cba0-4b52-bdd4-149114e9c202'}}} Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:18.463611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:18.465034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 400 to user: Invalid volume: volume d190657e-cba0-4b52-bdd4-149114e9c202 is already attached to instances: 0c861342-ab71-4ab6-bed1-41f3b6e38853 {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6760ead6-21b2-4170-a668-079219c57619 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.271887 Oct 16 01:02:18.466622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 311/591] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:02:18 2022] POST /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments => generated 174 bytes in 273 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:02:18.528428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:18.529014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:18.535609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.536082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.536514 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.537211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.537653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.538067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.672064 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'services' on Instance uuid 0c861342-ab71-4ab6-bed1-41f3b6e38853 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:02:18.748915 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4e4f5eb5-78e3-46f2-b687-742967e02f4b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments/d190657e-cba0-4b52-bdd4-149114e9c202" status: 202 len: 0 microversion: 2.1 time: 0.224689 Oct 16 01:02:18.749824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 281/592] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:02:18 2022] DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments/d190657e-cba0-4b52-bdd4-149114e9c202 => generated 0 bytes in 227 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:02:18.760117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:18.762303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:18.771118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.771622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.772069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.772783 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:18.773468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:18.773673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:18.808233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c1f4f2ab-954b-49e7-9f2a-e38a99eb8d6b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053925 Oct 16 01:02:18.808802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 312/593] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:02:18 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:19.822023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:19.824699 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:19.831493 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:19.831988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:19.832421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:19.833116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:19.833573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:19.834006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:19.863446 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-84c3e5e2-1375-455e-9904-d456fae71b04 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046271 Oct 16 01:02:19.864006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 282/594] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:02:19 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:02:19.928070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:19.929216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:19.936697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:19.937180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:19.937622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:19.939246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:19.939246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:19.939246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:19.965221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:02:20.043155 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6e76ba7e-ce84-4936-83db-8bea6ddd5f00 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 204 len: 0 microversion: 2.1 time: 0.118441 Oct 16 01:02:20.043776 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 313/595] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:02:19 2022] DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:02:20.052455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:20.053067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:20.059447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:20.060003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:20.060457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:20.061150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:20.061596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:20.062018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:20.203200 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:20.211534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7f8bb2b0-29f1-4142-ba00-c45051fbb77b tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1682 microversion: 2.1 time: 0.162797 Oct 16 01:02:20.212126 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 283/596] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:20 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1682 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:21.236197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:21.236905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:21.244167 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:21.244660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:21.245102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:21.245806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:21.246268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:21.246698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:21.385467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:21.393768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-20908a98-9af5-4a66-aae2-07614a725fdb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1682 microversion: 2.1 time: 0.161757 Oct 16 01:02:21.394628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 314/597] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:21 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:02:22.417191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:22.417807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:22.424387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:22.425428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:22.426140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:22.427137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:22.427805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:22.428450 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:22.663576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:22.672186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4b70f7a8-d279-4e0d-9889-b473d341bb50 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1682 microversion: 2.1 time: 0.259660 Oct 16 01:02:22.672832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 284/598] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:22 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1682 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:23.124146 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0c861342-ab71-4ab6-bed1-41f3b6e38853", "name": "network-vif-deleted", "tag": "35e81c79-fcfc-45cc-82e6-d74a12d99947"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:23.135961 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:23.136865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:23.137587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:23.151534 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] Creating event network-vif-deleted:35e81c79-fcfc-45cc-82e6-d74a12d99947 for instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 on hv-cinder-maste Oct 16 01:02:23.156620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-930fedd4-855b-45d0-8596-251995ce68b9 req-8e29c3b2-ca6b-482e-b276-17cf1f039fe7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036811 Oct 16 01:02:23.157614 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 315/599] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:23.696765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:23.697366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:23.704336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:23.704814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:23.705249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:23.705946 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:23.706405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:23.706811 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:23.738643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:23.739082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:23.739514 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:23.739955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'fault' on Instance uuid 0c861342-ab71-4ab6-bed1-41f3b6e38853 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:02:23.787301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:23.795453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1c62c0d5-60fc-4be0-8597-542256d91c83 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 200 len: 1332 microversion: 2.1 time: 0.103651 Oct 16 01:02:23.799358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 285/600] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:23 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 1332 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:24.820479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:24.821227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:24.827657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:24.828137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:24.828568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:24.829249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:24.829698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:24.830115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:24.843869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 HTTP/1.1 Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:02:24.845504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:02:24.847004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '0c861342-ab71-4ab6-bed1-41f3b6e38853'} Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:02:24.847857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:02:24.849391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:24.850755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:02:24.850755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-248f714c-52db-4690-9edf-8fd06533eda0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 404 len: 111 microversion: 2.1 time: 0.031269 Oct 16 01:02:24.850755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 316/601] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:24 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:02:24.857167 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-453ae0e0-6222-4b99-852b-57025a54dc81 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '.version_select at 0x7f8bbd9fd5e0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:24.876732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-453ae0e0-6222-4b99-852b-57025a54dc81 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1171493286" status: 202 len: 0 microversion: 2.1 time: 0.023535 Oct 16 01:02:24.877318 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 286/602] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:02:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1171493286 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:02:25.492291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b89e957a-bbea-427d-be6f-069571cba3b1 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: .version_select at 0x7f8bbd89c790>, body: {"keypair": {"name": "tempest-keypair-358106079", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNstpH+dM8Qo0zAGf7grkMQ9XaF4VQPhsZuScW9/zMl0a6GFj1r6tgNIy4vXi/UBJq0GxmavpCMlI5PLAuDQ2dPUAJ6Mo744Os75F+vG71mT9zNOasum3maylhUvySRzFw=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:25.508729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b89e957a-bbea-427d-be6f-069571cba3b1 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:02:25.512550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-b89e957a-bbea-427d-be6f-069571cba3b1 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:02:25.536194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b89e957a-bbea-427d-be6f-069571cba3b1 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.048273 Oct 16 01:02:25.536849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 317/603] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:02:25 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:26.826687 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-291090031", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7dae3d76-90a2-41d2-9dad-7f7856d5ab85"}], "security_groups": [{"name": "tempest-securitygroup--530173324"}], "key_name": "tempest-keypair-358106079", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:26.927410 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] validate_networks() for [('7dae3d76-90a2-41d2-9dad-7f7856d5ab85', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:02:27.294790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:02:27.295256 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:02:27.295733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:02:27.296139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:02:27.296577 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:02:27.305978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:27.306455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:27.306913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:27.307637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:27.308086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:27.308525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:27.325698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:02:27.329654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Getting quotas for user 466f2f8ddd184a98bdb5e922818905cc and project fb3c758e27684dcd9265e2d60036d80b. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:02:27.340708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:02:27.343030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:02:27.391315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ac59c17d-7ad6-4f68-8b11-1210e367e3e0 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.569517 Oct 16 01:02:27.392151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 287/604] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:02:26 2022] POST /compute/v2.1/servers => generated 401 bytes in 571 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:02:27.403790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-46019641-275d-4237-8ad8-4952fedbe5e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:27.405028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-46019641-275d-4237-8ad8-4952fedbe5e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:27.465715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-46019641-275d-4237-8ad8-4952fedbe5e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:27.472726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-46019641-275d-4237-8ad8-4952fedbe5e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1240 microversion: 2.1 time: 0.073325 Oct 16 01:02:27.473298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 318/605] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:27 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1240 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:28.496497 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:28.497102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:28.503152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:28.503634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:28.504061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:28.504742 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:28.505191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:28.505612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:28.572852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:28.581353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c8733aff-83ec-4f36-b4fe-c39ec047bcae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1292 microversion: 2.1 time: 0.089715 Oct 16 01:02:28.581922 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 288/606] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:28 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1292 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:29.605406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:29.605918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:29.612563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:29.613913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:29.614356 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:29.615288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:29.615746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:29.616415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:29.761777 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:29.770301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-04b3ae83-de51-449d-b9b7-cdf17a3560bd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.169658 Oct 16 01:02:29.770906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 319/607] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:29 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:30.293765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d627f49d-f82c-43c3-a5c5-b9ce678a168d", "tag": "55fcc1bb-db7a-478d-a43e-2302d6cbccc8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:30.301700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=50,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:30.311003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:30.311617 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:30.312128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:30.326842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] Creating event network-changed:55fcc1bb-db7a-478d-a43e-2302d6cbccc8 for instance d627f49d-f82c-43c3-a5c5-b9ce678a168d on hv-cinder-maste Oct 16 01:02:30.332809 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-d9b53f55-a9d0-41b7-8000-3013fc40cd81 req-87bbba78-0cb1-4d26-80a4-c9790825d75f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042943 Oct 16 01:02:30.333419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 289/608] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:30.793624 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:30.794264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:30.800766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:30.801244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:30.801677 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:30.802355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:30.802802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:30.803244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:30.938376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:30.946781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a0fbca14-adae-49fb-b37e-7e36676dc5ab tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.156733 Oct 16 01:02:30.947502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 320/609] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:30 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:31.971821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:31.976675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:31.986612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:31.987163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:31.987608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:31.988357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:31.988816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:31.989251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:32.193874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:32.203507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a712cec-6e2a-446b-87a6-097b7ffea6eb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.236189 Oct 16 01:02:32.204645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 290/610] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:31 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:33.233001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:33.233656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:33.240495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:33.240974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:33.241434 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:33.242142 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:33.242725 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:33.243855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:33.377421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:33.385975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8b78c3b8-b76e-40a8-a87c-79ed89b43c0e tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.156766 Oct 16 01:02:33.386566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 321/611] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:33 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:34.217621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d627f49d-f82c-43c3-a5c5-b9ce678a168d", "name": "network-vif-plugged", "status": "completed", "tag": "55fcc1bb-db7a-478d-a43e-2302d6cbccc8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:34.229029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:34.229660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:34.230109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:34.243190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] Creating event network-vif-plugged:55fcc1bb-db7a-478d-a43e-2302d6cbccc8 for instance d627f49d-f82c-43c3-a5c5-b9ce678a168d on hv-cinder-maste Oct 16 01:02:34.248871 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-e23768f0-fcaa-406f-83c8-8a90a956cbf4 req-56e666d7-4538-4d8b-8355-11ea9b81d8f2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034990 Oct 16 01:02:34.249473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 291/612] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:34.409593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:34.410230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:34.416973 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:34.417464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:34.417905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:34.418606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:34.419086 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:34.419517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:34.560376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:34.568451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-90e59761-a395-4ffe-bcdd-b3078884aae9 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.162760 Oct 16 01:02:34.569032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 322/613] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:34 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:35.084874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:35.085451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:35.092254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.092737 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.093168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.093879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.098728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.098728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.223642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lazy-loading 'services' on Instance uuid a14ab912-59d2-449f-b756-d19cad8d7b96 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:02:35.314471 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1c177973-143c-4b12-8012-4eb69f577820 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments/4ed0c81b-572c-4572-b561-c910c5cfac57" status: 202 len: 0 microversion: 2.1 time: 0.233789 Oct 16 01:02:35.315111 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 292/614] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:02:35 2022] DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments/4ed0c81b-572c-4572-b561-c910c5cfac57 => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:02:35.324562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:35.326705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:35.332774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.333260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.333705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.334403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.334953 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.335393 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.371264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f25d1367-6ec9-4588-8a68-1cfd45911099 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051594 Oct 16 01:02:35.372206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 323/615] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:02:35 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:35.592291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:35.592976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:35.599348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.599831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.600265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.600948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:35.601397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:35.601815 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:35.605138 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:35.746505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:35.755602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ed15f404-5228-4c72-ae36-2a4d48336582 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1365 microversion: 2.1 time: 0.167387 Oct 16 01:02:35.756194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 293/616] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:35 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:36.383488 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:36.385647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:36.392137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.392627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.393069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.393768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.394226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.394656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.425096 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d563e83f-90de-431b-876f-f7ec07464dfd tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045653 Oct 16 01:02:36.425681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 324/617] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:02:36 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:36.838269 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:36.838996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:36.844977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.845482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.845912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.846599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.847082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.847502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.877180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:36.879557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:36.881554 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=52,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:36.888080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.888836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.889458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.890348 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:36.890991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:36.891578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:36.895695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:36.987790 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:36.988456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-25b99d96-dd12-49b0-910f-257ee82d2aa2 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action" status: 202 len: 0 microversion: 2.1 time: 0.115705 Oct 16 01:02:36.989138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 325/618] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:02:36 2022] POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:02:36.997157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0c7f8281-14a4-44d6-94f7-96f21689a1cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1555 microversion: 2.1 time: 0.222275 Oct 16 01:02:36.997771 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 294/619] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:36 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1555 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:36.998339 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:36.999164 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:37.005456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.005946 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.006433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.007152 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.007611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.008580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.020625 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:37.022257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:37.030108 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.031031 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.031854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.032733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.033340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.033928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.146330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8f1db097-5674-4f9a-9c3a-377337d18a33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.129476 Oct 16 01:02:37.147456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 295/620] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:02:37 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-interface => generated 284 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:37.173261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:37.181809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8af3f484-ebce-4fb0-8571-f8af5fd21e50 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1749 microversion: 2.1 time: 0.188252 Oct 16 01:02:37.191114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 326/621] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:36 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1749 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:02:37.434411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:37.435330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:37.439655 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:37.441896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:37.450370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.450869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.452189 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.453746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.454270 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.454713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.485851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f2de1ed4-e0a9-487e-9ca7-1d55971c52ec tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052052 Oct 16 01:02:37.486439 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 296/622] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:02:37 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:02:37.564861 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:37.565427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:37.567781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:37.575054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.575545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.575984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.576686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.577140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.577568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.603433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:02:37.685428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b03e142e-37fa-44d6-83fc-9014d6af1b03 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 204 len: 0 microversion: 2.1 time: 0.125036 Oct 16 01:02:37.686823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 327/623] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:02:37 2022] DELETE /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:02:37.696352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:37.696940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:37.703632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.704121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.704559 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.705258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:37.705746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:37.706169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:37.948729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:37.957452 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9f7ecb9c-772e-41ef-ba74-221ab972560c tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1688 microversion: 2.1 time: 0.266057 Oct 16 01:02:37.958051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 297/624] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:37 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1688 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:02:38.206913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:38.207618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:38.218202 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:38.218730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:38.219212 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:38.219899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:38.220363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:38.220789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:38.377532 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:38.387217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d5514d85-9429-4ad6-bec8-e57c2759a0b4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1749 microversion: 2.1 time: 0.184733 Oct 16 01:02:38.388068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 328/625] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:38 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1749 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:38.618043 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d627f49d-f82c-43c3-a5c5-b9ce678a168d", "tag": "55fcc1bb-db7a-478d-a43e-2302d6cbccc8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:38.627480 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:38.627950 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:38.628374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:38.639971 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] Creating event network-changed:55fcc1bb-db7a-478d-a43e-2302d6cbccc8 for instance d627f49d-f82c-43c3-a5c5-b9ce678a168d on hv-cinder-maste Oct 16 01:02:38.644767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-fe3ec642-599b-4290-957c-40c00b924eef req-abacc739-4cf1-4fdd-aa4f-9d24bd70f0f9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030420 Oct 16 01:02:38.645530 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 298/626] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:02:38.981074 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:38.982333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:38.988986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:38.989470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:38.989905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:38.990609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:38.991091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:38.991518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:39.157661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:39.169165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-91bff1ed-ea5e-44b4-97b9-ae3d7f5a364b tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1688 microversion: 2.1 time: 0.192100 Oct 16 01:02:39.169760 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 329/627] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:38 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1688 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:39.412176 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:39.413042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:39.420403 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:39.420988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:39.421495 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:39.422272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:39.422845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:39.424359 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:39.581898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:39.591317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-def2ee3c-df74-4aa9-8ee6-3d8f32db7231 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1749 microversion: 2.1 time: 0.183803 Oct 16 01:02:39.591910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 299/628] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:39 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1749 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:40.193190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:40.193799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:40.201036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:40.201525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:40.201963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:40.202666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:40.203164 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:40.203593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:40.383585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:40.393017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a34cd682-7d3c-4f52-ade3-c7d316f0eded tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1688 microversion: 2.1 time: 0.203652 Oct 16 01:02:40.393615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 330/629] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:40 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1688 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:40.619306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:40.620250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:40.628868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:40.628868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:40.628868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:40.628868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:40.629355 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:40.630002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:40.821704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:40.832002 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-12f4ae67-49b0-4f6a-9e65-4241c321950d tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1749 microversion: 2.1 time: 0.217261 Oct 16 01:02:40.832418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 300/630] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:40 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1749 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:41.418456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:41.419113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:41.426104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:41.426608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:41.427089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:41.427811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:41.428271 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:41.429264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:41.645648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:41.664572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ebdb000c-4760-4191-bb31-577d7f7c1c0a tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1688 microversion: 2.1 time: 0.250685 Oct 16 01:02:41.665440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 331/631] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:41 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1688 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:41.859087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:41.859691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:41.866271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:41.866774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:41.867254 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:41.868042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:41.868451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:41.869485 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:42.224634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:42.236182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d7f943da-65fb-49d3-bc9b-dbdf9e5cb7ac tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.381986 Oct 16 01:02:42.236891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 301/632] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:41 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:42.694458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:42.695767 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:42.709909 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:42.711004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:42.711826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:42.713759 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:42.714673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:42.715528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:42.736787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a14ab912-59d2-449f-b756-d19cad8d7b96", "name": "network-vif-deleted", "tag": "eacd06b0-a0a4-40d4-b556-dd29e648da3c"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:42.747312 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:42.747802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:42.748498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:42.773683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] Creating event network-vif-deleted:eacd06b0-a0a4-40d4-b556-dd29e648da3c for instance a14ab912-59d2-449f-b756-d19cad8d7b96 on hv-cinder-maste Oct 16 01:02:42.782566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-b2d7c76a-2747-4c43-a91f-14549a137b33 req-cbbc511c-730a-4467-89a3-e1140e494ad9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049090 Oct 16 01:02:42.789035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 302/633] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:02:42.866457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:42.875640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1d818490-7151-494c-a3a6-a31700b89758 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 200 len: 1620 microversion: 2.1 time: 0.189350 Oct 16 01:02:42.876254 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 332/634] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:42 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 1620 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:43.262168 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:43.262975 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:43.269725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:43.270350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:43.271037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:43.272114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:43.272114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:43.272660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:43.445457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:43.454310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d7b8d62b-cfd9-4b7c-82d5-ca3ffd9cb989 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.197830 Oct 16 01:02:43.454933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 303/635] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:43 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:43.902509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:43.903177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: a14ab912-59d2-449f-b756-d19cad8d7b96] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:43.910074 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:43.910798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:43.911303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:43.912002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:43.912545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:43.913028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:43.927887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] HTTP exception thrown: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Request method failure captured: Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 HTTP/1.1 Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b-xNm9KCVjgM3wqxu3_jLffliMpX0rEBJ2pjnEEcTaHDwNHHNLws3WGBBUFguF0p4v0Uh6PcQ0OWevJVzKcLOchCd-DjlwU0EBAQ0HWFl9B8ZLgM7suC6TrG-0QTAhRmCIJzo4n5DR8fd3oJDcM_OYmQ6Ak0HH1w5jq7S_PXbqbcA Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:02:43.929661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5462fcbfca114c8885f77f42b479b13d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5462fcbfca114c8885f77f42b479b13d"}]}] Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: b01ce2d27bdb4c91a3f55c891f949e00 Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. Oct 16 01:02:43.932800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'a14ab912-59d2-449f-b756-d19cad8d7b96'} Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:02:43.934332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. Oct 16 01:02:43.937144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:43.940457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Returning 404 to user: Instance a14ab912-59d2-449f-b756-d19cad8d7b96 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:02:43.940457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-995c16d0-c3f2-42ad-84ce-b696f1e92e90 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96" status: 404 len: 111 microversion: 2.1 time: 0.033434 Oct 16 01:02:43.940457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 333/636] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:43 2022] GET /compute/v2.1/servers/a14ab912-59d2-449f-b756-d19cad8d7b96 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:02:43.947657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-536f14bc-1fdd-4cb4-bc60-2a88f781ae63 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '.version_select at 0x7f8bbd94c430>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:43.976558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-536f14bc-1fdd-4cb4-bc60-2a88f781ae63 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1657451180" status: 202 len: 0 microversion: 2.1 time: 0.038104 Oct 16 01:02:43.977074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 304/637] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:02:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1657451180 => generated 0 bytes in 40 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:02:44.483201 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:44.483823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:44.491296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:44.491779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:44.492355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:44.493224 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:44.493735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:44.494414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:44.667519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:44.681822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-53cb8f1f-af5b-4454-8ccf-a3a2b5a5b4d3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.202069 Oct 16 01:02:44.682722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 334/638] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:44 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:45.706769 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:45.707418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:45.716094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:45.716938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:45.717639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:45.718948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:45.719723 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:45.720477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:45.900541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:45.909514 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6809c104-94ca-4a9a-8704-0cdcb2d6b820 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.207603 Oct 16 01:02:45.910262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 305/639] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:45 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:46.938484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:46.939165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:46.946255 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:46.946743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:46.947561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:46.948580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:46.949036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:46.949467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:47.000420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:47.001442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:47.011013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:47.011502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:47.011937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:47.012686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:47.013140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:47.013595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:47.042061 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:02:47.126027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:47.137785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-75609190-5786-4e06-9db5-7b39b2311a74 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.203591 Oct 16 01:02:47.138388 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 335/640] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:46 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:47.147421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ea534a3d-37c5-4c05-91ef-feae2e692308 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 204 len: 0 microversion: 2.1 time: 0.153253 Oct 16 01:02:47.148314 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 306/641] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:02:46 2022] DELETE /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:02:47.160680 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:47.161300 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:47.170122 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:47.171071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:47.171872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:47.173185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:47.174048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:47.174845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:47.333227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:47.341986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2788f239-4ff5-4627-bb2f-356c8ccfabd2 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1508 microversion: 2.1 time: 0.187335 Oct 16 01:02:47.342566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 336/642] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:47 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1508 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:48.164166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:48.164910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:48.172646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:48.173143 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:48.173586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:48.174299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:48.174758 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:48.175210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:48.332596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:48.341466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f5d1cf59-09d1-44db-a37a-f1ae78e025a4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.181512 Oct 16 01:02:48.342073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 307/643] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:48 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:48.365469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:48.366053 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:48.373431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:48.373935 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:48.374383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:48.375190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:48.375795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:48.376431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:48.527825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:48.536084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f8bdb37e-df35-42ee-a587-7cdcb968fd10 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1508 microversion: 2.1 time: 0.174325 Oct 16 01:02:48.536794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 337/644] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:48 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1508 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:49.364297 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:49.364974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:49.372445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:49.372937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:49.373383 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:49.374094 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:49.374550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:49.375022 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:49.530480 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:49.539098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-034f6944-d429-44cf-b3d8-45aed7919d40 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.178525 Oct 16 01:02:49.539690 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 308/645] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:49 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:49.559580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:49.560279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:49.566784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:49.567306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:49.567746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:49.568450 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:49.568912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:49.569376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:49.723630 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:49.732070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fd62a1b2-d6cb-4102-a849-390b8f346887 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1508 microversion: 2.1 time: 0.176342 Oct 16 01:02:49.732640 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 338/646] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:49 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1508 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:02:50.155287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9dffef0-6081-49ec-9c9e-2412f35c0907", "name": "network-vif-deleted", "tag": "e6727a07-61b9-4fef-83e4-902cce857bee"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:50.167315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.167794 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.168229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.182472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] Creating event network-vif-deleted:e6727a07-61b9-4fef-83e4-902cce857bee for instance b9dffef0-6081-49ec-9c9e-2412f35c0907 on hv-cinder-maste Oct 16 01:02:50.188041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-d93962bb-540f-4dfa-ac00-71febadbe41e req-90118285-0398-475e-9b4a-f06676b85aff service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036398 Oct 16 01:02:50.188652 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 309/647] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:02:50.562505 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:50.563213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:50.570114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.570609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.571186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.571930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.572386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.573057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.720526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:50.723140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:50.731723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-caa3f194-96a8-4d11-a2ba-898cd0dc1617 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.173700 Oct 16 01:02:50.732320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 339/648] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:50 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:50.760373 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:50.761109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:50.767509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.767972 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.768344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.769118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.769543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.769978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.803676 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:50.804170 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:50.804805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:50.805260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lazy-loading 'fault' on Instance uuid b9dffef0-6081-49ec-9c9e-2412f35c0907 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:02:50.853945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:50.862416 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-156bab03-8c92-4e7e-bafa-7a02fc4a29ce tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 200 len: 1312 microversion: 2.1 time: 0.107024 Oct 16 01:02:50.863045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 310/649] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:50 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 1312 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:51.389545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:51.389979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:51.395934 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.396420 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.396859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.397549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.398009 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.398435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.421884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-985ff0da-2d5e-40c2-8c57-9c0d8d0295dc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/ips" status: 200 len: 150 microversion: 2.20 time: 0.091232 Oct 16 01:02:51.422488 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 340/650] 10.222.0.25 () {60 vars in 1289 bytes} [Sun Oct 16 01:02:51 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/ips => generated 150 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 01:02:51.756902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:51.757693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:51.764632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.765860 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.765860 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.766259 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.766717 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.767175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.770286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:02:51.887915 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:51.888535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: b9dffef0-6081-49ec-9c9e-2412f35c0907] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:51.895999 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.896498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.896969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.897708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.898187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.898617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.914933 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] HTTP exception thrown: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Request method failure captured: Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 HTTP/1.1 Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b-xNm9KCVjgM3wqxu3_jLffliMpX0rEBJ2pjnEEcTaHDwNHHNLws3WGBBUFguF0p4v0Uh6PcQ0OWevJVzKcLOchCd-DjlwU0EBAQ0HWFl9B8ZLgM7suC6TrG-0QTAhRmCIJzo4n5DR8fd3oJDcM_OYmQ6Ak0HH1w5jq7S_PXbqbcA Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:02:51.917675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5462fcbfca114c8885f77f42b479b13d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5462fcbfca114c8885f77f42b479b13d"}]}] Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: b01ce2d27bdb4c91a3f55c891f949e00 Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. Oct 16 01:02:51.923056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'b9dffef0-6081-49ec-9c9e-2412f35c0907'} Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:02:51.924653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. Oct 16 01:02:51.927089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:02:51.928913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Returning 404 to user: Instance b9dffef0-6081-49ec-9c9e-2412f35c0907 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:02:51.928913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3e4a197b-8571-4b15-9885-9a89d8f8d6b4 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907" status: 404 len: 111 microversion: 2.1 time: 0.035330 Oct 16 01:02:51.928913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 341/651] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:51 2022] GET /compute/v2.1/servers/b9dffef0-6081-49ec-9c9e-2412f35c0907 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 16 01:02:51.930153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:51.931925 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:51.932438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:51.939148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.939736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.940182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.940798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-abcd4bd1-5f3b-43a9-a17b-f2d388c47644 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.188293 Oct 16 01:02:51.940931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:51.941701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 311/652] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:51 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:51.941888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:51.941888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:51.973472 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:02:52.057132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1cc15cd7-aebf-408b-bf27-4685b4069a98 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "DELETE /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 204 len: 0 microversion: 2.1 time: 0.132582 Oct 16 01:02:52.058029 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 342/653] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:02:51 2022] DELETE /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:02:52.067547 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:52.068513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:52.075065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:52.075556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:52.075996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:52.076705 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:52.077166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:52.077594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:52.239259 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:52.248054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4324a4a8-2a06-4307-a153-a6cb0ee4ecfb tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.185146 Oct 16 01:02:52.248670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 312/654] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:52 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:02:52.967743 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:52.968460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:52.974920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:52.975408 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:52.975838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:52.976526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:52.976976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:52.977394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.139996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:53.148720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-11bb0218-3ba5-4d62-992d-fd9221d0f37a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.185131 Oct 16 01:02:53.149343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 343/655] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:52 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:53.271693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:53.272419 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:53.279919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.280413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.280847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.281549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.282001 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.282422 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.345897 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:53.346345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:53.352477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.353181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.353737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.354445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.354933 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.355367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.379114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lazy-loading 'flavor' on Instance uuid 0f07f47f-8851-4e67-9c77-d4115ff157c7 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:02:53.433517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:53.441959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-29849bc2-9843-484f-bbc4-ec2150b78358 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.174052 Oct 16 01:02:53.442590 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 313/656] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:53 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:53.608412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c400d858-6696-403e-af0d-331736d53f2d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/action" status: 202 len: 0 microversion: 2.20 time: 0.267355 Oct 16 01:02:53.609062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 344/657] 10.222.0.25 () {62 vars in 1319 bytes} [Sun Oct 16 01:02:53 2022] POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/action => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:02:53.617882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:53.619147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:53.625223 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.625719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.626152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.626883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:53.627343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:53.627775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:53.761629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:53.769752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:53.771231 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:53.772430 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b30c0228-0388-48a6-9041-efe7f022d166 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.159128 Oct 16 01:02:53.772999 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 314/658] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:53 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:02:54.174471 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:54.175214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:54.182728 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.183262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.183704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.184470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.184928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.185390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.333892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:54.343383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-acb65d1d-215e-4a85-93f9-e94fb21959d0 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.173514 Oct 16 01:02:54.343985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 345/659] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:54 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:54.481374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:54.481973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:54.488992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.489485 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.489928 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.490647 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.491145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.491575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.647686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:54.656992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a7e87df4-7a0b-4e33-beb6-a6c8626d8964 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.180061 Oct 16 01:02:54.657583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 315/660] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:54 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:54.798303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:54.798983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:54.805250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.805751 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.806191 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.807058 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:54.807523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:54.807984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:54.948708 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:54.957921 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:54.959549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:54.960737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-eedfaead-f575-4033-b9e7-23d79ecd75c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.166341 Oct 16 01:02:54.961345 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 346/661] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:54 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:55.377722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:55.378321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:55.384411 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.384888 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:55.385317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:55.386006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.386456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:55.386886 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:55.613691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:55.622324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9be4423d-4ec8-4e9b-8117-e2022be6f1df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.248690 Oct 16 01:02:55.628534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 316/662] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:55 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:55.680752 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:55.681556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:55.688467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.688967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:55.689414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:55.690131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.690611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:55.691062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:55.848232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b01ce2d27bdb4c91a3f55c891f949e00', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5462fcbfca114c8885f77f42b479b13d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:55.860172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e3618da1-9267-411b-948e-086a114c9ed9 tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 200 len: 1521 microversion: 2.1 time: 0.183026 Oct 16 01:02:55.861135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 347/663] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:55 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:55.987754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:55.988380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:55.997610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.998101 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:55.998482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:55.999253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:55.999712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.000138 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.100399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8d7ed11a-f7aa-4bed-bf41-596a5feb1c14", "name": "network-vif-deleted", "tag": "37a89ecc-65c3-483a-b753-8ddbd32b4132"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:02:56.110394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:56.111397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.111603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.124354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] Creating event network-vif-deleted:37a89ecc-65c3-483a-b753-8ddbd32b4132 for instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 on hv-cinder-maste Oct 16 01:02:56.130085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-5cdb14b5-f729-4f85-a53b-ed1f90301e72 req-d1245dac-4038-45c8-b6c2-d71929ed5e7c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099907 Oct 16 01:02:56.130904 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 348/664] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:02:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:02:56.154258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:56.160874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:56.162438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:56.163753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e123ab88-7776-4456-b949-283570e3d3f3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.180034 Oct 16 01:02:56.164412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 317/665] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:55 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:02:56.646456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:56.647170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:56.653825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:56.654321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.654763 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.655510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:56.655977 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.656410 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.829123 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:56.837615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8b9d9c21-cf78-4198-8bd3-dcaf4b4773f4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.195622 Oct 16 01:02:56.838190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 349/666] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:56 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:56.894977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:56.895677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] [instance: 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:56.902768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:56.903197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.903630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.904339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:56.904789 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:56.905212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:56.920455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] HTTP exception thrown: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Request method failure captured: Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 HTTP/1.1 Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b-xNm9KCVjgM3wqxu3_jLffliMpX0rEBJ2pjnEEcTaHDwNHHNLws3WGBBUFguF0p4v0Uh6PcQ0OWevJVzKcLOchCd-DjlwU0EBAQ0HWFl9B8ZLgM7suC6TrG-0QTAhRmCIJzo4n5DR8fd3oJDcM_OYmQ6Ak0HH1w5jq7S_PXbqbcA Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:02:56.922121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5462fcbfca114c8885f77f42b479b13d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5462fcbfca114c8885f77f42b479b13d", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5462fcbfca114c8885f77f42b479b13d"}]}] Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 5462fcbfca114c8885f77f42b479b13d Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1271918241 Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: b01ce2d27bdb4c91a3f55c891f949e00 Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1271918241-project Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. Oct 16 01:02:56.923967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '8d7ed11a-f7aa-4bed-bf41-596a5feb1c14'} Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:02:56.924904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. Oct 16 01:02:56.926426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:02:56.928122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] Returning 404 to user: Instance 8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:02:56.928122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-626741c4-dfa8-4322-8fa8-4ab28b9b7fae tempest-ServerRescueNegativeTestJSON-1271918241 tempest-ServerRescueNegativeTestJSON-1271918241-project] 10.222.0.25 "GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14" status: 404 len: 111 microversion: 2.1 time: 0.032685 Oct 16 01:02:56.928122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 318/667] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:56 2022] GET /compute/v2.1/servers/8d7ed11a-f7aa-4bed-bf41-596a5feb1c14 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:02:57.194788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:57.195749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:57.202700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:57.203235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:57.203678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:57.204382 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:57.204838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:57.205267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:57.440831 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:57.450003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:57.451506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:57.452738 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-89fe6851-c134-405f-9dfb-abcff56004d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.262068 Oct 16 01:02:57.453349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 350/668] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:57 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:57.866492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:57.867787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:57.878717 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:57.879668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:57.880432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:57.881675 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:57.882498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:57.883299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:58.050212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:58.061207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9ef0fd7d-2387-4ffc-b76a-5014b804b4e7 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.202288 Oct 16 01:02:58.074603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 319/669] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:57 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:58.481746 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:58.482396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:58.489397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:58.490117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:58.490609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:58.491545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:58.492173 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:58.492740 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:58.705786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:58.716438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:58.718063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:58.719803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6e0a3b3b-b0c9-4639-8e51-a3b720ad3dfe tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.243121 Oct 16 01:02:58.721272 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 351/670] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:58 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:02:59.090788 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:59.092107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:59.101295 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:59.102372 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:59.102594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:59.103707 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:59.104390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:59.105091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:59.416026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:59.434151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-047dff93-5495-433f-bbd8-a04674ee2786 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.350265 Oct 16 01:02:59.434750 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 320/671] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:02:59 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:02:59.750206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:02:59.751323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:02:59.757838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:59.758446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:59.759210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:59.759931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:02:59.760403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:02:59.760862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:02:59.942664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:59.958890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:59.962893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:02:59.965085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-be2c460b-030f-445f-9da3-dd54fcaa887c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.218524 Oct 16 01:02:59.966671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 352/672] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:02:59 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:00.458572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:00.459162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:00.465952 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:00.466733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:00.467192 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:00.467907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:00.468354 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:00.468953 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:00.652988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:00.661250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0c45d8e7-11ae-4a9d-b95a-a1e08e058f10 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.206751 Oct 16 01:03:00.661832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 321/673] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:00 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:00.994296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:00.994959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:01.003190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:01.003901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:01.004333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:01.006015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:01.010367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:01.010914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:01.264315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:01.273609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:01.276262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:01.277445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-be1f14ae-f465-445f-acaf-91e3868ecc01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.288130 Oct 16 01:03:01.289234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 353/674] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:00 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:01.685813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:01.686415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:01.692765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:01.693248 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:01.693687 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:01.694631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:01.695115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:01.695544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:01.900824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:01.916155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2a1ef627-f0b1-4c26-bbe9-9b47ddd0f8df tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1754 microversion: 2.1 time: 0.233035 Oct 16 01:03:01.916803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 322/675] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:01 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1754 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:02.302892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:02.303511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:02.311031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:02.311526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:02.311964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:02.312670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:02.313127 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:02.313555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:02.330211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:02.468515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:02.478325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:02.479279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:02.480476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-efca483c-dea4-48a2-8f05-6f5e54844ef7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.181590 Oct 16 01:03:02.481065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 354/676] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:02 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:02.941255 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:02.942578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:02.944983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=76,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:02.951654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:02.952153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:02.952589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:02.953582 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:02.954065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:02.954658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:03.115568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:03.125842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-11b1d379-39a1-44a2-b723-fc6fd40e55ad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1757 microversion: 2.1 time: 0.188559 Oct 16 01:03:03.126429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 323/677] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:02 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1757 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:03.507653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:03.509273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:03.516671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:03.517307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:03.517753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:03.518608 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:03.519124 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:03.519650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:03.797157 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:03.812296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:03.814097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:03.816520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b6ece682-4fa7-45c0-903d-b3d48b43d66a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.314643 Oct 16 01:03:03.817860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 355/678] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:03 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:04.868262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:04.869502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:04.877278 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:04.877775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:04.878209 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:04.879332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:04.880110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:04.880543 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:05.118954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:05.131518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:05.133964 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:05.135551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ccd49c5c-7fdd-4d31-8708-a96f2aa3ba50 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.271512 Oct 16 01:03:05.137183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 324/679] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:04 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:05.987628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:05.988916 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:05.999894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.000882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.002256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.004381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.005496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.006520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.055002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:03:06.154790 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-49788900-24e1-44ba-b045-b1c70e717aa7 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "DELETE /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 204 len: 0 microversion: 2.1 time: 0.175740 Oct 16 01:03:06.155460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 356/680] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:05 2022] DELETE /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:03:06.168374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:06.169730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:06.171426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:06.172237 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:06.178369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.179207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.179643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.180786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.181190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.181317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.181767 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.182202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.182541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:06.183061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:06.183276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.183498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:06.372636 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:06.382584 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:06.384905 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:06.387427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0c9f0094-7433-4e01-b639-df9422afb57f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.223349 Oct 16 01:03:06.403375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 357/681] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:06 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:06.441290 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:06.450407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cfac65c3-7aaa-4999-840f-13c8c7d68d22 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1576 microversion: 2.1 time: 0.288789 Oct 16 01:03:06.451277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 325/682] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:06 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1576 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:07.107086 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3939a1d9-5907-46f5-a292-008d102650d1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: .version_select at 0x7f8bbd822430>, body: {"keypair": {"name": "tempest-keypair-1914944121", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBF/PwE5k6p+ivmhyFpbDdijjiMIhDK8HfxaZQfgctcyK5woY422YiOtfkdBFMGwylN7BRTkDlUwFc5WHBlSv4/vqC44pYhAvmthFRbafgzjlnGHIEiAkzSGA4xKAAu/nTw=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:07.113290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:07.127575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-3939a1d9-5907-46f5-a292-008d102650d1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for project b8852bfe31aa4ba68190661d26c177cb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:03:07.131906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-3939a1d9-5907-46f5-a292-008d102650d1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for user e1283100ac2c439cb308b9c57ae31f40 and project b8852bfe31aa4ba68190661d26c177cb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:03:07.157266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3939a1d9-5907-46f5-a292-008d102650d1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.210216 Oct 16 01:03:07.157893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 358/683] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:03:06 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:07.419986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:07.422454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:07.432874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:07.433814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:07.434513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:07.435464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:07.436163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:07.436591 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:07.476672 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:07.477855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:07.495714 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:07.496374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:07.496853 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:07.497648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:07.498130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:07.500555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:07.702555 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:07.724720 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:07.732680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:07.732680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b40665fa-1abe-457a-8bc8-9de431f5a8bc tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.317527 Oct 16 01:03:07.732680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 326/684] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:07 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:08.030731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:08.044871 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e2e35e50-6a32-4025-93b5-712742e2de82 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1576 microversion: 2.1 time: 0.573247 Oct 16 01:03:08.045652 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 359/685] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:07 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1576 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:08.758499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:08.759660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:08.770712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:08.771954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:08.772968 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:08.775136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:08.776252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:08.777260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:09.011936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:09.026677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:09.028195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:09.029422 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c9651296-cfde-4e48-821b-17167aab85c5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.278778 Oct 16 01:03:09.038812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 327/686] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:08 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:09.070552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:09.071332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:09.085095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:09.086548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:09.090350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:09.093519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:09.107738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e232da69-f419-4e6d-802e-a253ad2587ab", "name": "network-vif-deleted", "tag": "1cc7e38e-e043-419a-986d-342c38dba97f"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:09.108998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:09.111413 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:09.131112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:09.131990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:09.132662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:09.154381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] Creating event network-vif-deleted:1cc7e38e-e043-419a-986d-342c38dba97f for instance e232da69-f419-4e6d-802e-a253ad2587ab on hv-cinder-maste Oct 16 01:03:09.163240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-c0acf012-01dc-4e33-b14d-ad0a82475d70 req-15cfc6f2-eda7-4487-a2ee-65e24ec0186d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059339 Oct 16 01:03:09.166745 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 328/687] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:03:09.195515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1110879998", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "57335160-65ed-4241-a862-dbf37da52fd2"}], "security_groups": [{"name": "tempest-securitygroup--269443264"}], "key_name": "tempest-keypair-1914944121", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:09.207579 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:09.230511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9c95917e-3165-4514-8bf8-65b2da8b909b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1508 microversion: 2.1 time: 0.163448 Oct 16 01:03:09.232461 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 360/688] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:09 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1508 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:09.356891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] validate_networks() for [('57335160-65ed-4241-a862-dbf37da52fd2', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:03:09.974085 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:03:09.978997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:03:09.978997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:03:09.978997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:03:09.978997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:03:09.988304 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:09.989276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:09.989853 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:09.990939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:09.991534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:09.992104 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.005177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=100 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:10.018035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for project b8852bfe31aa4ba68190661d26c177cb. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:03:10.024635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for user e1283100ac2c439cb308b9c57ae31f40 and project b8852bfe31aa4ba68190661d26c177cb. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:03:10.039756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:03:10.042979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:03:10.055722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:10.056655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:10.063941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:10.064757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:10.065384 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.066306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:10.066979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:10.067571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.105019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f8c1978a-cd35-4f8f-b50c-1de9e8f21d3b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.936560 Oct 16 01:03:10.106032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 329/689] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:03:09 2022] POST /compute/v2.1/servers => generated 401 bytes in 939 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:03:10.124947 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-459490d7-7d2d-4921-aa12-7b5860c47a65 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:10.125546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-459490d7-7d2d-4921-aa12-7b5860c47a65 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:10.206280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-459490d7-7d2d-4921-aa12-7b5860c47a65 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:10.228450 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-459490d7-7d2d-4921-aa12-7b5860c47a65 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1238 microversion: 2.1 time: 0.107368 Oct 16 01:03:10.229080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 330/690] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:10 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1238 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:10.256441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:10.257279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:10.265412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:10.266163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:10.266759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.267907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:10.269062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:10.269669 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.341944 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:10.353144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:10.353870 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:10.354684 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:10.355336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lazy-loading 'fault' on Instance uuid e232da69-f419-4e6d-802e-a253ad2587ab {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:03:10.357017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:10.360985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:10.362477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-403399b5-1f07-498b-8fff-073ce70c5000 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.311239 Oct 16 01:03:10.375531 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 361/691] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:10 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:10.427000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:10.435098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e7a08717-b7c6-4118-ad54-d1dae946c21d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 200 len: 1183 microversion: 2.1 time: 0.182345 Oct 16 01:03:10.436817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 331/692] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:10 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 1183 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:11.253069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:11.253772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:11.261085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.261585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.262028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.262741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.263238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.263670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.354721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:11.365137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6cbe3543-6146-4fad-a544-95b930a5d1b1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1290 microversion: 2.1 time: 0.115861 Oct 16 01:03:11.379824 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 362/693] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:11 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1290 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:11.390333 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:11.391569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:11.400009 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.400550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.400991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.401777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.402257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.402728 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.467517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:11.468754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:11.480152 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.481691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.482442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.484069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:11.484818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:11.485512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:11.509349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] HTTP exception thrown: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Request method failure captured: Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab HTTP/1.1 Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1eG2j1-Qx4qQKlflgNnJ9o38mFemkYM9ej90vtQx4uBAfNoUgZe7Ad3_Gzmuz7XTwYgWUxXLq9IS5Op3iLbi9EUTnQj-YCg4Dc7hD7cRcPJpMaZq-g3E1x85xxWVY1RJ1q7Y9J8JNMmZVQ3NEpMJG2ruwTnOIZZdu0eVzhJOk8c_1c Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:03:11.511422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cb1743385ea14cc39244c83e8fc88e7e", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cb1743385ea14cc39244c83e8fc88e7e"}]}] Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: c4e65cef89af48a7a51e5548634de75f Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:03:11.512892 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'e232da69-f419-4e6d-802e-a253ad2587ab'} Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:11.513695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:03:11.514958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:11.516161 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Returning 404 to user: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:11.516161 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b08381e4-70c4-46e1-abac-7b6f47d138d2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 404 len: 111 microversion: 2.1 time: 0.057771 Oct 16 01:03:11.526522 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 363/694] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:11 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:11.636037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:11.647626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:11.649610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:11.651076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5bf18f26-f532-4c48-aaaf-5ffbea7c8967 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1742 microversion: 2.20 time: 0.265300 Oct 16 01:03:11.658980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 332/695] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:11 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1742 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:11.698603 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-421627063", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-466177183", "security_groups": [{"name": "tempest-secgroup-smoke-1119436186"}], "block_device_mapping_v2": [{"uuid": "0bac125a-0773-44b5-92f7-7b1e03420312", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "67400858-429f-4052-a9a5-3012690406d5"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:11.861651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] validate_networks() for [('67400858-429f-4052-a9a5-3012690406d5', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:03:12.402051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:12.403408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:12.413420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.416637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.417102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.419038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.420339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.420986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.478346 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:03:12.479446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:03:12.480044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:03:12.480527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:03:12.481015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:03:12.491902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.492737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.493362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.494128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.494588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.495042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.515299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:03:12.518382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:12.521007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for user c4e65cef89af48a7a51e5548634de75f and project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:03:12.529423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-868d8e56-0e3d-48fd-b3d3-52027e65e01e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1296 microversion: 2.1 time: 0.134850 Oct 16 01:03:12.537751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 333/696] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:12 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1296 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:12.602422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:03:12.605186 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='0bac125a-0773-44b5-92f7-7b1e03420312',volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:03:12.677889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:12.678634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:12.686303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.687135 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.687818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.688742 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:12.689388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:12.689983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:12.778220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7d35c8a3-d87a-4b4f-bc4f-d811baa031dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.084535 Oct 16 01:03:12.779576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 364/697] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:03:11 2022] POST /compute/v2.1/servers => generated 406 bytes in 1087 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:03:12.791620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2161ba1b-ecf4-4539-9814-5a17bea0c3fc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:12.792502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2161ba1b-ecf4-4539-9814-5a17bea0c3fc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:12.860738 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2161ba1b-ecf4-4539-9814-5a17bea0c3fc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:12.870188 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2161ba1b-ecf4-4539-9814-5a17bea0c3fc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1093 microversion: 2.1 time: 0.083086 Oct 16 01:03:12.883371 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 365/698] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:12 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1093 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:12.968465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:12.974275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:12.980191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:12.982356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e2acaf4a-996e-4bc2-9a10-133d5a6fd878 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1742 microversion: 2.20 time: 0.308522 Oct 16 01:03:12.983609 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 334/699] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:12 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1742 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:13.562351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:13.563795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:13.572991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:13.573839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:13.574520 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:13.575932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:13.576818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:13.577694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:13.780833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:13.799389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-022b035e-fdeb-4bf4-8cf1-e8ac3cd9eec7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.241881 Oct 16 01:03:13.800135 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 366/700] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:13 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:13.896729 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:13.897643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:13.905714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:13.906219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:13.908132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:13.908747 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:13.910163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:13.912332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.014305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:14.015379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:14.024732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.025552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.026225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.027222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.028646 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.030441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.111509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:14.124813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7c9bf2f7-b67b-4186-bffc-ed037422057b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1191 microversion: 2.1 time: 0.234969 Oct 16 01:03:14.143455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 335/701] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:13 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1191 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:14.286406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:14.298575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:14.301511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:14.303529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e40b9493-c7da-455e-87d0-fa6162aeb863 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.293937 Oct 16 01:03:14.305048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 367/702] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:14 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:14.359633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3182305-f754-4b0f-989d-7362967cb10a", "tag": "01028fe7-7644-4a03-bb34-6762d2043774"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:14.384831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.385323 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.385761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.428255 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] Creating event network-changed:01028fe7-7644-4a03-bb34-6762d2043774 for instance f3182305-f754-4b0f-989d-7362967cb10a on hv-cinder-maste Oct 16 01:03:14.443606 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-19e5691a-6c74-4823-b598-5c1563d9a164 req-45853796-6562-45fa-a8fc-dc7bae8cd0ae service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089185 Oct 16 01:03:14.444861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 336/703] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:14.768138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "name": "network-vif-plugged", "status": "completed", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:14.779491 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.788530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.789855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.802778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] Creating event network-vif-plugged:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:03:14.809579 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-59c198a3-11d8-421b-9954-3058fc948f5e req-7a9c50ea-4964-498a-b29c-52919d1e2535 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.205233 Oct 16 01:03:14.810545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 368/704] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:14.828938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:14.832813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:14.845468 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.846369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.846946 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:14.847725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:14.848387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:14.849032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:15.158500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:15.160432 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:15.183881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:15.184715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:15.185372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:15.186513 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:15.187296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:15.187928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:15.307631 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:15.320503 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-27ac8fbe-c46e-4d87-ad70-4c0b8a196a65 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1209 microversion: 2.1 time: 0.169487 Oct 16 01:03:15.321518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 369/705] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:15 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1209 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:15.337918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:15.339053 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:15.352206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:15.352785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:15.353498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:15.354396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:15.355128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:15.356260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:15.486239 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:15.503141 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e4a9d9f6-fdb4-488d-8a32-b088689ac3f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.683212 Oct 16 01:03:15.508919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 337/706] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:14 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:15.583860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:15.595733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:15.597678 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:15.599622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c94a95d7-3467-422b-b9af-81ef2bddb77d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.269465 Oct 16 01:03:15.600373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 370/707] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:15 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:16.348734 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:16.349496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:16.357563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.358147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.358765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.359700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.360582 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.361150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.535030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:16.535649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:16.539364 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:16.547999 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.550457 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.551307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.552305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.553519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.554173 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.554301 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0ea1a2ac-531a-4ffa-8517-b1345ad1aeaa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1277 microversion: 2.1 time: 0.207548 Oct 16 01:03:16.560986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 338/708] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:16 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1277 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:16.629851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:16.629851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:16.647499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.647499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.647499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.653716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.654187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.654582 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.831097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:16.840613 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dd528b6e-ca5e-48ab-96ed-6b71fa2eb65b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.311079 Oct 16 01:03:16.856324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 371/709] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:16 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:16.869109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:16.871046 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:16.878140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.878629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.879108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.879813 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:16.880266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:16.880687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:16.927607 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0e8f60a8-df45-47ae-ae25-b778f07757fb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/ips" status: 200 len: 149 microversion: 2.1 time: 0.064768 Oct 16 01:03:16.927912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 372/710] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:03:16 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/ips => generated 149 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:17.042872 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:17.092214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:17.094369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:17.096761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-73a2da7e-9af3-4419-8f02-b70bab5ac46a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.471893 Oct 16 01:03:17.110889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 339/711] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:16 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:17.582838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:17.583787 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:17.592545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:17.593059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:17.593721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:17.594881 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:17.595347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:17.595775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:17.789691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:17.792469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:17.804405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f9e82d2b-47ed-46aa-9f8a-fc3a6ff0658c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.226046 Oct 16 01:03:17.805617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 373/712] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:17 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:17.866841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:17.867923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:17.875202 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:17.876612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:17.877258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:17.883585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:17.884417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:17.885045 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.125757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:18.126433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:18.132516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.133183 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.133778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.134488 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.134974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.135406 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.289727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:18.306321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f499bc85-8899-4d99-893e-60d31cd9bd4d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.444382 Oct 16 01:03:18.312998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 340/713] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:17 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:18.368479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:18.378731 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:18.380386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:18.381592 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ea28829d-b8eb-490f-b5a5-755a8b488e32 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.260086 Oct 16 01:03:18.382607 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 374/714] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:18 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:18.424408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "391edabf-85a9-4565-92f6-ebd0664d3e20", "tag": "ea701c3f-eb65-4a82-83d0-b757b0f39524"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:18.449232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.451296 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.463779 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.476035 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d5aa4b8e-91a7-4124-8908-a19129767e4f"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:18.479803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:18.485205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] Creating event network-changed:ea701c3f-eb65-4a82-83d0-b757b0f39524 for instance 391edabf-85a9-4565-92f6-ebd0664d3e20 on hv-cinder-maste Oct 16 01:03:18.491447 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-87867f52-18a9-45f2-a4e1-4b068af4d527 req-14cc025b-bf66-4e47-9c74-52e8db8c3e89 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070576 Oct 16 01:03:18.494752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 341/715] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:18.495315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.496081 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.496745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.497794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.498474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.499181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.832172 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:18.834473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:18.845195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.845943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.846911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.848091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:18.849029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:18.849684 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:18.890615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9134dfba-b430-4201-84dd-ee004659c9dd tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "POST /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.417191 Oct 16 01:03:18.891375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 375/716] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:03:18 2022] POST /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments => generated 194 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:03:19.297515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:19.309096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-22e6f268-6e2f-435c-9112-ef8830bc7088 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.482243 Oct 16 01:03:19.324119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 342/717] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:18 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:19.348332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:19.349524 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:19.373337 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:19.374012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:19.374724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:19.375964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:19.376748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:19.377282 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:19.411990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:19.413042 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:19.424583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:19.425340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:19.435211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:19.436166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:19.437025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:19.437480 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:19.641325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:19.649087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:19.657750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-74f07128-295d-4226-97c7-5ff8c9d47d8d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.324752 Oct 16 01:03:19.658236 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:19.658611 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 376/718] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:19 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:19.659700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:19.660869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-240bbe45-652d-43b9-b5a7-9b9e123ee31d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.256497 Oct 16 01:03:19.676603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 343/719] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:19 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:20.333932 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:20.335230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:20.344494 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.345277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.345880 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.346761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.347405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.347822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.383478 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:20.385806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:20.399353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:20.403062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=8 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:20.407990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.410355 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.412159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.413704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.415997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.420967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.484026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lazy-loading 'flavor' on Instance uuid 2f92209b-6feb-4e65-af10-6872f24bbc95 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:03:20.584672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ea336b57-91ea-4d3e-aaa9-9284448707cd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "POST /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95/action" status: 202 len: 0 microversion: 2.1 time: 0.206573 Oct 16 01:03:20.590132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 344/720] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:03:20 2022] POST /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95/action => generated 0 bytes in 213 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:03:20.600054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3182305-f754-4b0f-989d-7362967cb10a", "name": "network-vif-plugged", "status": "completed", "tag": "01028fe7-7644-4a03-bb34-6762d2043774"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:20.600816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:20.610989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.611478 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.611914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.620738 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-55933413-91cc-4993-bf0d-c146335026aa tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.290707 Oct 16 01:03:20.622666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 377/721] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:20 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:20.629959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] Creating event network-vif-plugged:01028fe7-7644-4a03-bb34-6762d2043774 for instance f3182305-f754-4b0f-989d-7362967cb10a on hv-cinder-maste Oct 16 01:03:20.641741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:20.643553 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:20.651650 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-7ba9bf08-a6e2-4d78-8349-675f3917f36a req-60e0ae9d-b858-469c-bb33-309c1d9afe62 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059110 Oct 16 01:03:20.660216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 345/722] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:20.669415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.669908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.670347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.671122 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.671586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.672526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.689633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:20.690653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:20.705064 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.705799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.706492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.707873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.709800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.710334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.883577 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:20.894431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ebb6c442-e9bb-4a43-9899-0cd75aab9cca tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.270226 Oct 16 01:03:20.895658 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 378/723] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:20 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:20.901119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:20.902261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:20.913427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.914943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.915693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.916661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:20.917306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:20.917901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:20.986644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:20.996643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:20.998226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:20.999498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-03e88e5e-8a72-4ad8-93ac-d316f2dc111f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.315075 Oct 16 01:03:21.011111 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 346/724] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:20 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:21.123963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:21.134307 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5e2db9d3-dbbd-4c2f-b540-b078f75b2ba4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.237253 Oct 16 01:03:21.135496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 379/725] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:20 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:21.650923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:21.651550 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:21.658313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:21.658800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:21.659261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:21.659985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:21.660437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:21.660894 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:21.665465 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:21.807825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:21.816277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8ec38894-ce2c-413b-9331-b8495f864b53 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.169854 Oct 16 01:03:21.816858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 347/726] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:21 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:21.921832 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:21.923198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:21.930589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:21.931102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:21.931536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:21.932270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:21.932719 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:21.933143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.028029 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:22.028830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:22.038337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.039420 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.043235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.044139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.046916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.046916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.134183 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:22.143992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e74d26d8-b2fd-426f-8551-bbf4076d13d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.226587 Oct 16 01:03:22.144958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 380/727] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:21 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:22.166679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:22.167435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:22.183029 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.186012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.186855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.188018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.189075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.189754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.215283 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:22.229228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:22.231434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:22.233551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-21c4a812-6ea5-4e6f-a4d4-aa6d8520b12a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.209743 Oct 16 01:03:22.234897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 348/728] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:22 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:22.365084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:22.375373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-55528c43-5a27-4b36-bc80-f87d7ab5b738 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1363 microversion: 2.1 time: 0.219169 Oct 16 01:03:22.404648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 381/729] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:22 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1363 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:22.840252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:22.841504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:22.858073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.859846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.860634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:22.861637 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:22.862906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:22.863843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.022610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.032551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-66d161b8-5c5a-4f4c-b39e-fe9d608f6bd2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.197392 Oct 16 01:03:23.047504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 349/730] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:22 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:23.169818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:23.171181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:23.177144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:23.177906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=12 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:23.180100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.181094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.181620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.182668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.183162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.183600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.265896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:23.267140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:23.276364 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.276673 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.277063 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.280084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.280755 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.282620 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.434191 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.451559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-72bbc537-2b2e-4fdf-aee1-608f864fe2e5 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.285613 Oct 16 01:03:23.453585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 382/731] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:23 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:23.463036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:23.464314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:23.470973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.473112 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.473653 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.474234 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.474757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.475702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.475702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.485230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.486923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.488267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f13b89fb-354e-4c5c-9b5c-3b9edf3bc069 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.226533 Oct 16 01:03:23.507522 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 350/732] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:23 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:23.682385 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:23.694568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fb4d25d6-d4a6-4c24-8f97-995310b60e01 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1550 microversion: 2.1 time: 0.239772 Oct 16 01:03:23.711697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 383/733] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:23 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1550 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:23.718777 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:23.719289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:23.725667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.726814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.727506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.728205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:23.728662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:23.729082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:23.839199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-32e4fdf6-fe40-498f-b585-cb3f6b15c290 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.124002 Oct 16 01:03:23.847023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 351/734] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:03:23 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-interface => generated 285 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:24.057727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:24.058347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:24.067185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.067687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.068124 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.068847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.069300 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.070384 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.252077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:24.262098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9c3da892-e8ee-43cc-9eae-1f35e363c1a3 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.208583 Oct 16 01:03:24.263181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 384/735] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:24 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:24.475371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:24.476927 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:24.501109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.501740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.502319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.503276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.504065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.504813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.523468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:24.524331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:24.537489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.537994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.538594 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.540226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:24.541016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:24.541621 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:24.742101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:24.752555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:24.755604 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:24.757587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d468e44a-7c3c-4f0f-864c-c6738e168fcb tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.240634 Oct 16 01:03:24.758772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 385/736] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:24 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:24.835056 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:24.847869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cafc0c4b-72f4-4aac-bbaa-308c215d14f7 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.376910 Oct 16 01:03:24.864628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 352/737] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:24 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:25.293241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:25.294377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:25.301340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.301819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.302456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.304237 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.305568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.306850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.488732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:25.497538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7c438d2e-3b37-4950-b624-58adf4ad5cf6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.210336 Oct 16 01:03:25.498169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 386/738] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:25 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:25.526619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f3182305-f754-4b0f-989d-7362967cb10a", "tag": "01028fe7-7644-4a03-bb34-6762d2043774"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:25.536801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.537437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.537878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.551716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] Creating event network-changed:01028fe7-7644-4a03-bb34-6762d2043774 for instance f3182305-f754-4b0f-989d-7362967cb10a on hv-cinder-maste Oct 16 01:03:25.557229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-b08964f6-8323-4fa2-8691-53d5a394381c req-1741ec98-432a-459d-a246-371c95f9dc3e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035065 Oct 16 01:03:25.557840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 353/739] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:25.785504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:25.786149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:25.792797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.793207 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.793917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.794618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.795052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.795519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.892251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:25.892962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:25.900561 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.901074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.901515 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.902242 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:25.902700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:25.903175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:25.978722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:25.990060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:25.992218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:25.994032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-146421f2-d348-4dd5-8a7e-43b82e2b3d14 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1737 microversion: 2.20 time: 0.212464 Oct 16 01:03:25.994828 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 387/740] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:25 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1737 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:26.055356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:26.063846 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f5d29864-c1a7-4a23-ad2f-f97e07909320 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.176284 Oct 16 01:03:26.064540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 354/741] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:25 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:26.527255 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:26.535641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:26.547963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:26.548762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:26.549383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:26.550885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:26.551635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:26.553650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:26.725479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:26.743625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-af426115-1695-431b-b339-259a83b32af4 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.226079 Oct 16 01:03:26.744227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 388/742] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:26 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:27.021798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:27.023076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:27.031385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.032344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.033267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.034268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.037656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.039137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.088548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:27.089460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:27.096896 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.097684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.098397 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.099249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.099827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.100336 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.106109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:27.278734 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:27.290515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-43c79b3b-57ae-4b00-800a-03679a0abfc1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.206762 Oct 16 01:03:27.291284 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 389/743] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:27 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:27.392597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:27.403337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:27.405293 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:27.406787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f7f31d31-cea3-4cab-b24e-af1674a9100f tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1719 microversion: 2.20 time: 0.390728 Oct 16 01:03:27.407581 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 355/744] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:27 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1719 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:27.776943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:27.780031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:27.792781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.793691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.794675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.796184 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:27.796990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:27.797762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:27.979693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:27.990590 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a9a4b7e5-900b-4a08-b813-9cb0d5882073 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.222078 Oct 16 01:03:27.991799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 390/745] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:27 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:28.026287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:28.027264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:28.040073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.040943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.041791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.043228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.044100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.044904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.138156 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lazy-loading 'services' on Instance uuid d627f49d-f82c-43c3-a5c5-b9ce678a168d {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:03:28.217228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f9cda3c2-e13e-4d40-aa66-f3a5df2fde69 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments/d5aa4b8e-91a7-4124-8908-a19129767e4f" status: 202 len: 0 microversion: 2.1 time: 0.199484 Oct 16 01:03:28.217882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 356/746] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:03:28 2022] DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments/d5aa4b8e-91a7-4124-8908-a19129767e4f => generated 0 bytes in 201 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:03:28.230553 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:28.233748 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:28.240474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.240966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.241466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.242335 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.242761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.243216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.277688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2c19a561-297a-45b8-ae22-4a1857e0abb7 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054461 Oct 16 01:03:28.278291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 391/747] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:28 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:28.321848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:28.323213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:28.330814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.331337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.331782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.332490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.332944 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.333363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.438540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:28.439833 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:28.450431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.451512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.452403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.453758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.454676 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.455666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.550727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:28.567661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f2b9c39b-3009-419d-b01e-5413d72b381c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.252208 Oct 16 01:03:28.568825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 357/748] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:28 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:28.652522 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:28.666096 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:28.668692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "391edabf-85a9-4565-92f6-ebd0664d3e20", "name": "network-vif-plugged", "status": "completed", "tag": "ea701c3f-eb65-4a82-83d0-b757b0f39524"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:28.668986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:28.672060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ac8572c7-2b80-4f0f-a0c0-43f196fd3438 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1719 microversion: 2.20 time: 0.240887 Oct 16 01:03:28.680433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 392/749] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:28 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1719 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:28.683821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:28.684499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:28.685004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:28.700761 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] Creating event network-vif-plugged:ea701c3f-eb65-4a82-83d0-b757b0f39524 for instance 391edabf-85a9-4565-92f6-ebd0664d3e20 on hv-cinder-maste Oct 16 01:03:28.705874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-b537707b-1fb3-45db-baca-c1ba0c8defaa req-9ab12613-6fe3-4e71-91a4-40ff8f532b26 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048484 Oct 16 01:03:28.706558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 358/750] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:29.016773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:29.017336 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:29.024549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.025044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.025486 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.026196 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.026660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.027125 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.176779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:29.185864 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-89035341-ce8d-4b87-a1c4-d82e1f606ddc tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.172975 Oct 16 01:03:29.186538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 393/751] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:29 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:29.295266 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:29.297616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:29.304879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.305468 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.305908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.306631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.307115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.307538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.355316 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f8802133-4ae0-4a1c-a5cc-928eefdba7bf tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065531 Oct 16 01:03:29.355925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 359/752] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:29 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:29.598816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:29.598816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:29.600309 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:29.612330 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.613503 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.614246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.615804 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.616837 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.618825 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.700273 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:29.700919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:29.709918 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.711242 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.712310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.713272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:29.713732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:29.714365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:29.837952 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:29.846846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1c387833-fd8c-4786-991b-bbf7e6373aad tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.253288 Oct 16 01:03:29.847468 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 394/753] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:29 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:29.916802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:29.927992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:29.931153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:29.932178 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-95a85692-4df3-4295-8348-5fd51d1b3ec7 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1719 microversion: 2.20 time: 0.236167 Oct 16 01:03:29.932791 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 360/754] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:29 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1719 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:30.217056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:30.218786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:30.225347 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.225845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.226299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.227054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.227516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.227946 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.374818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:30.376820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:30.384896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.385409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.385856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.386583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.387073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.388025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.399510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:30.412596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8a5df514-892a-43bd-9bef-1ad05b993776 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1265 microversion: 2.1 time: 0.201781 Oct 16 01:03:30.412596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 395/755] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:30 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1265 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:30.434600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ba2a39ef-159d-4c7d-931c-26c2ec29c83c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066537 Oct 16 01:03:30.434907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 361/756] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:30 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:03:30.730037 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "name": "network-vif-unplugged", "status": "completed", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:30.740091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.740596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.741040 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.755222 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] Creating event network-vif-unplugged:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:03:30.760894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-e35644df-1bc6-4510-8442-486391612d56 req-dd7b0a5b-5b2f-4f1f-9604-7b244411e5f3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035060 Oct 16 01:03:30.761519 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 396/757] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:30 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:30.870824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:30.871451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:30.877892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.878380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.878833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.879567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.880010 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.880429 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.962649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:30.963476 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:30.970084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.970583 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.971071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:30.971779 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:30.972325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:30.973982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.032564 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.041593 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-03d57b71-85bd-4001-a756-0b018858f20e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.174651 Oct 16 01:03:31.042218 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 362/758] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:30 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:31.125823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.134480 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.135987 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.137228 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-54474cbd-222e-480d-ab5d-19022ee61757 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1722 microversion: 2.20 time: 0.181714 Oct 16 01:03:31.137775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 397/759] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:30 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1722 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:31.163035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "41e2283a-837c-4e36-937d-49de9af964a0"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:31.166166 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.168173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:31.173632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.174117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.174551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.175284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.175738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.176159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.356157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:31.437064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.437730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.464240 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.464970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.466705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.474840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.476778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.477271 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.491221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8cb84d64-26a4-4fbe-9e83-e68284ca55b5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.331939 Oct 16 01:03:31.491941 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 363/760] 10.222.0.25 () {62 vars in 1364 bytes} [Sun Oct 16 01:03:31 2022] POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments => generated 188 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 16 01:03:31.496984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.499762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.509372 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.510488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.510959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.511682 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.512144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.512579 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.546507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-28c3a33a-f468-49bc-ba4c-58c8707fc187 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053604 Oct 16 01:03:31.547132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 364/761] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:31 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:03:31.622387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.623032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.629201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.629765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.630386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.631315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.631894 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.632960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.649130 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.656942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-649705fb-e535-4cf3-947c-f05bf37b0091 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1451 microversion: 2.1 time: 0.226191 Oct 16 01:03:31.657730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 398/762] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:31 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1451 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:31.658908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:03:31.662723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.663306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.669601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.670107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.670545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.671269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.671721 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.672138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.736162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-29ce53d4-7adc-46c4-a413-ed4e9477ff71 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 204 len: 0 microversion: 2.1 time: 0.117558 Oct 16 01:03:31.736852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 365/763] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:31 2022] DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:03:31.742719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.743363 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.749317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.749800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.750267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.750995 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.751451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.751880 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.929730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dc88e57b-f8e0-495c-a888-d97c95e1090e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments/41e2283a-837c-4e36-937d-49de9af964a0" status: 202 len: 0 microversion: 2.20 time: 0.270443 Oct 16 01:03:31.930510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 399/764] 10.222.0.25 () {60 vars in 1457 bytes} [Sun Oct 16 01:03:31 2022] DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments/41e2283a-837c-4e36-937d-49de9af964a0 => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:03:31.938826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:31.939485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:31.948305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.949085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.949697 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.950695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:31.951349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:31.951942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:31.979685 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:31.988399 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-fead2c70-c682-465c-afb2-a7f2db0fbec0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1451 microversion: 2.1 time: 0.250320 Oct 16 01:03:31.989708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 366/765] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:31 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1451 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:32.007553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:32.008499 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:32.018220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.018710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.019451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.020236 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.020630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.021321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.073443 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119013) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 16 01:03:32.150425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:32.154792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-827e52ff-9577-4d6c-ad22-5318a2080a61 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/action" status: 202 len: 0 microversion: 2.20 time: 0.151288 Oct 16 01:03:32.155589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 367/766] 10.222.0.25 () {62 vars in 1319 bytes} [Sun Oct 16 01:03:32 2022] POST /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:03:32.166562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3934fc00-2599-4025-9d80-1a7a05c139e5 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1679 microversion: 2.1 time: 0.234615 Oct 16 01:03:32.167096 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:32.168102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:32.175679 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.176439 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.177080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.177974 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.178764 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.180122 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.180288 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 400/767] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:31 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1679 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:32.192131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:32.192745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:32.201442 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.202149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.202590 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.203327 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:32.203789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:32.204223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:32.346626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:32.355500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4702d12d-64f5-4598-a257-a03e7117a3ea tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.195414 Oct 16 01:03:32.357053 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 368/768] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:32 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:32.459068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:32.469720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:32.471671 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:32.473147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ebefb6e4-3f43-4d26-89fa-3a6260a98b29 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1670 microversion: 2.20 time: 0.291662 Oct 16 01:03:32.474037 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 401/769] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:32 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1670 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:33.196911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:33.197776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:33.204519 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.204993 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.205636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.206445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.207268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.207708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.379508 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:33.380452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:33.387595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.388094 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.388533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.389411 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.390033 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.390625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.409332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:33.422191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-25f9fa75-57b3-4dfa-b757-fb2fb94d112c tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1679 microversion: 2.1 time: 0.229172 Oct 16 01:03:33.432925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 369/770] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:33 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1679 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:33.502774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:33.503516 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:33.511741 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.512382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.512920 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.513732 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:33.514291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:33.514818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:33.817340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:33.826687 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:33.828174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:33.829361 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-bb7ebf5b-9178-437a-87e9-611ee2d19c5c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1672 microversion: 2.20 time: 0.331518 Oct 16 01:03:33.830189 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 370/771] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:33 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1672 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:33.917280 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:33.925835 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d778d540-b58b-415c-9905-aafead60c154 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.550683 Oct 16 01:03:33.926609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 402/772] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:33 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:34.282358 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "391edabf-85a9-4565-92f6-ebd0664d3e20", "tag": "ea701c3f-eb65-4a82-83d0-b757b0f39524"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:34.294405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.294969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.295404 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.307856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] Creating event network-changed:ea701c3f-eb65-4a82-83d0-b757b0f39524 for instance 391edabf-85a9-4565-92f6-ebd0664d3e20 on hv-cinder-maste Oct 16 01:03:34.312938 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-ff37bb29-b15f-416d-abbf-1589d738cdfa req-b1ca92bb-c1ee-4d84-99bc-62ddbe5fb4d5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.109679 Oct 16 01:03:34.314897 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 371/773] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:34.451026 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:34.452509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:34.461663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.462296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.462981 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.463745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.464217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.464967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.555504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:34.559336 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:34.564985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-64ddf586-2275-44cf-b013-5aecdcfadd1f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1612 microversion: 2.1 time: 0.122921 Oct 16 01:03:34.574925 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 403/774] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:34 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1612 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:34.783401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d627f49d-f82c-43c3-a5c5-b9ce678a168d", "name": "network-vif-deleted", "tag": "55fcc1bb-db7a-478d-a43e-2302d6cbccc8"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:34.793321 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.793798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.794221 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.806284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] Creating event network-vif-deleted:55fcc1bb-db7a-478d-a43e-2302d6cbccc8 for instance d627f49d-f82c-43c3-a5c5-b9ce678a168d on hv-cinder-maste Oct 16 01:03:34.811231 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-08ad7dd5-53f2-43b4-b918-44dd57f379db req-adc24ac7-af7f-4a81-9ee8-b8d0dcf4bfd0 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031859 Oct 16 01:03:34.811826 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 372/775] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:34.856965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:34.858068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:34.865819 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.867886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.868346 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.869084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.869546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.869976 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.950982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:34.951692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:34.963044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.965163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.967394 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:34.969756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:34.972071 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:34.972698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:35.073755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:35.087926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:35.089398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:35.090663 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ea93ff71-9a99-4e7e-a841-deb02968b7a0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.237993 Oct 16 01:03:35.091291 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 404/776] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:34 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:35.189274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:35.198836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-89f2324d-5c7d-4645-ad25-e47a633275d1 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.252434 Oct 16 01:03:35.199464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 373/777] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:34 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:35.598506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:35.603685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:35.620807 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:35.621537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:35.621969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:35.622679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:35.623163 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:35.623585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:35.912250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '466f2f8ddd184a98bdb5e922818905cc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb3c758e27684dcd9265e2d60036d80b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:35.921320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d8476218-944e-4cd3-b05a-ae128102f288 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 200 len: 1325 microversion: 2.1 time: 0.331132 Oct 16 01:03:35.922060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 405/778] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:35 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 1325 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:36.120328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:36.121080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:36.131619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.132115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.132590 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.133401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.133847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.134277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.222502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:36.223148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:36.229763 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.231063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.231511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.232215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.232665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.233085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.372153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:36.379913 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ef73692e-c520-4411-b378-fa4f3c3e46d3 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.161341 Oct 16 01:03:36.381219 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 406/779] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:36 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:36.383355 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:36.392891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:36.394457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:36.395680 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7eec8e12-ada0-4027-9bdc-757eb15bae8a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.281620 Oct 16 01:03:36.404020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 374/780] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:36 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:36.948704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:36.949233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:36.955724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.956207 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.956650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.957343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:36.957800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:36.958256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:36.973075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d HTTP/1.1 Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:03:36.974745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:36.976722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'd627f49d-f82c-43c3-a5c5-b9ce678a168d'} Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:36.977669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:36.979301 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:36.981602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:36.981602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b1204469-80f6-4aae-ba29-e9326e34958f tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 404 len: 111 microversion: 2.1 time: 0.032381 Oct 16 01:03:36.981602 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 407/781] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:36 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:36.985585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-45202cf9-ab27-4d7a-a1b3-0d2d4e3a3cae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '.version_select at 0x7f8bbd94c550>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:37.006174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-45202cf9-ab27-4d7a-a1b3-0d2d4e3a3cae tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-358106079" status: 202 len: 0 microversion: 2.1 time: 0.024271 Oct 16 01:03:37.007409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 375/782] 10.222.0.25 () {58 vars in 1218 bytes} [Sun Oct 16 01:03:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-358106079 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:03:37.408266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:37.408926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:37.415915 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.416615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.417233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.418355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.419085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.419673 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.426303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:37.427128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:37.436464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.437817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.438781 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.439678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.440350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.440950 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.617225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:37.627095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e6d35554-df22-436c-8876-4120a68f9209 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1509 microversion: 2.1 time: 0.224394 Oct 16 01:03:37.628108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 408/783] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:37 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1509 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:37.661959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:37.662812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:37.675171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.675692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.676136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.677053 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.683879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.685504 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.747772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=119012) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 16 01:03:37.787723 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:37.799963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:37.804616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:37.807565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4fa76bdf-f03c-495d-af3f-5c9f76c955b1 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.387693 Oct 16 01:03:37.811416 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 376/784] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:37 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:37.901316 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-67406345-b929-4d55-a807-2e5e32d3ee9a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "POST /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95/action" status: 202 len: 0 microversion: 2.1 time: 0.243332 Oct 16 01:03:37.908785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 409/785] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:03:37 2022] POST /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95/action => generated 0 bytes in 250 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:03:37.913487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:37.914308 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:37.920789 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.921937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.922634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:37.923638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:37.924328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:37.925132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:38.197445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:38.206597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-efc4bae5-41e7-475f-9b20-45bc89252064 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.297465 Oct 16 01:03:38.207730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 377/786] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:37 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:38.842783 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:38.843545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:38.850914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:38.851682 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:38.852274 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:38.853141 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:38.853757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:38.854197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.002319 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:39.011786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:39.013260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:39.014440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7537a749-d2d0-4c22-afe4-e4122d4dccae tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.176270 Oct 16 01:03:39.015064 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 410/787] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:38 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:39.242317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:39.242317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:39.242317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.242317 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.243199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.243199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.243199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.243199 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.246615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:39.247515 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:39.254368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.255156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.255627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.256367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.257436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.257872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.258030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d HTTP/1.1 Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:39.259956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.261533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'd627f49d-f82c-43c3-a5c5-b9ce678a168d'} Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:39.262434 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.263825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.268331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:39.268331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d0d96aef-1cb6-4304-aeb5-64fa15f7b3ca tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 404 len: 111 microversion: 2.1 time: 0.038009 Oct 16 01:03:39.268331 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 378/788] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:39 2022] DELETE /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:39.284032 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:39.284643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: d627f49d-f82c-43c3-a5c5-b9ce678a168d] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:39.291190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.291736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.292121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.292832 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:39.293291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:39.293717 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:39.301381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:39.314573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d HTTP/1.1 Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:39.316536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.317981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'd627f49d-f82c-43c3-a5c5-b9ce678a168d'} Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:39.319608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. Oct 16 01:03:39.321007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:39.322538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance d627f49d-f82c-43c3-a5c5-b9ce678a168d could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:39.322538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-816d3024-a2e0-4082-a364-995bf7861218 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d" status: 404 len: 111 microversion: 2.1 time: 0.048971 Oct 16 01:03:39.322538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 379/789] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:39 2022] GET /compute/v2.1/servers/d627f49d-f82c-43c3-a5c5-b9ce678a168d => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:39.443097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:39.456321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2e412213-b09a-41be-bbf2-a81cf7999b37 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.220255 Oct 16 01:03:39.457523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 411/790] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:39 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:40.042406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.043986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.051879 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.052366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.052805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.053652 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.054320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.054752 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.200189 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:40.210051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:40.211873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:40.213072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-95f8a4ea-903a-4dab-b1ae-c4f10722ba7d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.176340 Oct 16 01:03:40.218906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 380/791] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:40 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:40.483930 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.484549 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.486424 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "name": "network-vif-unplugged", "status": "completed", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:40.491578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.491962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.492403 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.493111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.493643 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.494067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.496843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.497444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.498021 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.511538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.compute.server_external_events [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] Unable to find a host for instance 2f92209b-6feb-4e65-af10-6872f24bbc95. Dropping event network-vif-unplugged {{(pid=119013) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 16 01:03:40.512471 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-428fee90-8392-4c6c-9fd2-3638b70a1384 req-f0fdd644-5160-48ec-aa8d-39edfc9b1e01 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.029768 Oct 16 01:03:40.513425 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 381/792] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:40 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 31 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Oct 16 01:03:40.518548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.519074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.525513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.526028 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.526463 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.527400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.527949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.528472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.577446 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 HTTP/1.1 Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:40.579086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.580558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '0c861342-ab71-4ab6-bed1-41f3b6e38853'} Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:40.581926 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.585382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.588526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:40.589877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8b420e84-4742-4b99-a6c3-7998f221c4df tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 404 len: 111 microversion: 2.1 time: 0.075131 Oct 16 01:03:40.590904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 382/793] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:40 2022] DELETE /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:40.599509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.600431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 0c861342-ab71-4ab6-bed1-41f3b6e38853] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.607005 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.607599 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.607937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.608654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.609110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.609563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.626735 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 HTTP/1.1 Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:40.628336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.629736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '0c861342-ab71-4ab6-bed1-41f3b6e38853'} Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:40.630578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. Oct 16 01:03:40.632904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.634081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 0c861342-ab71-4ab6-bed1-41f3b6e38853 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:40.634081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-90f2c84f-4c43-4ba2-87af-4c8f0f9e4a57 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853" status: 404 len: 111 microversion: 2.1 time: 0.034784 Oct 16 01:03:40.634292 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 383/794] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:40 2022] GET /compute/v2.1/servers/0c861342-ab71-4ab6-bed1-41f3b6e38853 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:40.639588 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.640413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.654789 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.655623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.656258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.657177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.657801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.658406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.683348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe HTTP/1.1 Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:40.684970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.686624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '18a404c6-15f6-40d3-bfbc-df301770a4fe'} Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:40.690667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.692388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.694469 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:40.694469 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-121ad973-1c69-4f48-aa58-526e7ad97b30 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 404 len: 111 microversion: 2.1 time: 0.050688 Oct 16 01:03:40.694469 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 384/795] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:40 2022] DELETE /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:40.701258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:40.701258 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: 18a404c6-15f6-40d3-bfbc-df301770a4fe] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:40.703897 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:40.711187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.711841 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.712379 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.713193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:40.713351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9ce592a8-6ae8-4637-b45b-0ee2fa9946ec tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.236833 Oct 16 01:03:40.713826 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:40.714909 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:40.719880 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 412/796] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:40 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:40.737537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe HTTP/1.1 Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:40.739389 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.741713 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '18a404c6-15f6-40d3-bfbc-df301770a4fe'} Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:40.742490 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. Oct 16 01:03:40.743738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:40.746281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance 18a404c6-15f6-40d3-bfbc-df301770a4fe could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:40.747425 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5e03e338-696b-4452-88ed-f78c376548d2 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe" status: 404 len: 111 microversion: 2.1 time: 0.054293 Oct 16 01:03:40.748630 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 385/797] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:40 2022] GET /compute/v2.1/servers/18a404c6-15f6-40d3-bfbc-df301770a4fe => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:41.242232 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:41.242846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:41.250436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.251466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.252165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.253343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.254333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.255299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.450945 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:41.461639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:41.464022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:41.466293 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ead60469-f3cd-4e5a-b056-242fdec4f6d3 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.228105 Oct 16 01:03:41.487121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 413/798] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:41 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:41.739151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:41.739834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:41.750634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.751161 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.751602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.752308 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.752756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.753178 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.918389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:41.919381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:41.933467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.934656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.935713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.937233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.938312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.939374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.942431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:41.952960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-24d41f9c-246a-4fa0-ab40-11fcb582143b tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.217900 Oct 16 01:03:41.953945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 386/799] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:41 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:41.965901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 HTTP/1.1 Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:03:41.968762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:41.971138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'f0dbf503-273a-4834-af7e-20ee0ac8b573'} Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:41.972332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:41.974068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:03:41.976512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:41.977448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d48ab6ec-84d8-4fc8-b8cc-94bb30f00d33 tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 404 len: 111 microversion: 2.1 time: 0.066434 Oct 16 01:03:41.978216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 414/800] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:41 2022] DELETE /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:41.986276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:41.986890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] [instance: f0dbf503-273a-4834-af7e-20ee0ac8b573] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:41.993088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.993536 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.994335 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:41.995799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:41.996267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:41.996711 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.011172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-932958323"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:42.011861 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:42.014306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] HTTP exception thrown: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Request method failure captured: Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 HTTP/1.1 Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1b_FKx10CkhlIOY-6a54-6mlW38gfBbula7YyNykhdppyssqz1NETFNnqk9-LVswB6z-Ri1OVjoJn93dbBbhQiXrwzFnG8Y-3FhndKfQUFdrQKYyCINh61LsxvWzVIoUXd_0ZBz0NQBaat4FBTc-j35R4wGVQ74oKQ19gsDWl7dkqk Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:42.016152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/fb3c758e27684dcd9265e2d60036d80b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_fb3c758e27684dcd9265e2d60036d80b", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/fb3c758e27684dcd9265e2d60036d80b"}]}] Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: fb3c758e27684dcd9265e2d60036d80b Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428578509 Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 466f2f8ddd184a98bdb5e922818905cc Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeNegativeTest-428578509-project Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:42.017823 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'f0dbf503-273a-4834-af7e-20ee0ac8b573'} Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:42.018848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. Oct 16 01:03:42.020845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:42.023018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] Returning 404 to user: Instance f0dbf503-273a-4834-af7e-20ee0ac8b573 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:42.023018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7ad5b310-3a5b-4f83-b3e1-70e07e57f7cb tempest-AttachVolumeNegativeTest-428578509 tempest-AttachVolumeNegativeTest-428578509-project] 10.222.0.25 "GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573" status: 404 len: 111 microversion: 2.1 time: 0.035248 Oct 16 01:03:42.023018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 387/801] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:41 2022] GET /compute/v2.1/servers/f0dbf503-273a-4834-af7e-20ee0ac8b573 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:42.029366 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.029910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:42.030359 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.031659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.031659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:42.031996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.223201 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.network.security_group_api [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Removing security group ef62311a-4a30-42aa-bd68-cc2deadeb13a from port c570709d-29cc-4c4f-a39b-9bfd4da608ed Oct 16 01:03:42.495405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:42.496415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:42.504279 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.505565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:42.506232 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.507212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.507837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:42.508494 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.705016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a2cf3408-bd7b-485f-8281-4b79788e09ab tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action" status: 202 len: 0 microversion: 2.1 time: 0.698536 Oct 16 01:03:42.711694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 415/802] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:03:42 2022] POST /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/action => generated 0 bytes in 706 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:03:42.778918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:42.790477 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.791424 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:42.792120 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:42.807247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:42.816101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] Creating event network-changed:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:03:42.821959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:42.823731 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:42.824500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-3efe9629-81bb-4ecf-84e1-1294821891ae req-ca5c8276-2b0e-458e-9c30-bf8bffd8bdf1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050641 Oct 16 01:03:42.824930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-465ac4e7-2662-4fdc-ba1d-4b23e0a48d93 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.335096 Oct 16 01:03:42.826396 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 416/803] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:42.835401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 388/804] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:42 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:42.982830 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:42.983586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:42.997492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:42.997990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:43.003090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:43.003906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:43.004398 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:43.004837 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:43.244872 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:43.257268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-aebcc1ee-7b3e-4a47-b440-91bb3ee7c4e8 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.280535 Oct 16 01:03:43.273780 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 417/805] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:42 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:43.859973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:43.861280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:43.879431 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:43.887715 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:43.888336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:43.889865 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:43.890517 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:43.893587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.285656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:44.286781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:44.300072 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.303498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.304256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.305031 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.307351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.307805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.509808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:44.526228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:44.529354 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:44.535636 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0eeeb599-19ad-46a6-a4b8-ef8ff2ac4823 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.682139 Oct 16 01:03:44.546899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 389/806] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:43 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:44.554954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:44.554954 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:44.580216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.589504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.590368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.591322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.593400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.594033 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.607303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:44.616149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e71c6d2a-5027-40d9-a3de-41ac813f5d96 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.334280 Oct 16 01:03:44.638137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 418/807] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:44 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:44.739228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lazy-loading 'services' on Instance uuid aa091bd0-0127-4e6a-8b16-a7956cc5379e {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:03:44.877278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:44.901365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a21d6dec-f62d-4cc7-8588-0c7152a46da9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments/992410fd-c57f-46d6-8158-3244996d4182" status: 202 len: 0 microversion: 2.1 time: 0.361038 Oct 16 01:03:44.904951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 390/808] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:03:44 2022] DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments/992410fd-c57f-46d6-8158-3244996d4182 => generated 0 bytes in 367 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:03:44.910286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.911244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.911858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.917385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:44.930099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:44.946312 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] Creating event network-changed:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:03:44.948857 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.949816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.950671 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.953767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:44.954496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:44.955128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:44.961368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-fbdc9d57-bdfb-4d2b-b130-cb2e08e5c1a3 req-7f35e6b2-6117-4d38-94b1-6f098b21c8a0 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097601 Oct 16 01:03:44.963389 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 419/809] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:45.039783 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c1e650ab-dd66-4877-8d7c-77b36a1684da tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.131116 Oct 16 01:03:45.040304 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 391/810] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:44 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 197 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:45.571912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:45.573796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:45.588952 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:45.589735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:45.591473 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:45.595108 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:45.596051 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:45.597635 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:45.646516 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:45.647387 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:45.655751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:45.662948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:45.665838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:45.666642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:45.671700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:45.672574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:45.939981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:45.949455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8176d4d1-d009-4129-a879-78c666dad2bc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.307032 Oct 16 01:03:45.950050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 392/811] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:45 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:46.051717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:46.052175 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:46.055734 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:46.068265 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:46.070268 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:46.073249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:46.073935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:46.074590 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:46.075283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d0dfe689-9cfb-46db-aed6-688fcb679d3c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.510741 Oct 16 01:03:46.089754 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 420/812] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:45 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:46.090004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:46.090004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:46.090004 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:46.152012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-193aefc6-903b-4bd2-a225-e2b558f19d53 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.103203 Oct 16 01:03:46.169198 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 393/813] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:46 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 197 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:46.973226 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:46.979214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:46.987063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:46.988182 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:46.988991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:46.989767 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:46.991147 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:46.991668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:47.107672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:47.108291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:47.114670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:47.115595 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:47.116531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:47.117538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:47.118197 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:47.118834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:47.172485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:47.181367 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ef3f3bc2-ae82-4065-801e-08edac446f67 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.213389 Oct 16 01:03:47.182372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 421/814] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:46 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:47.191369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:47.195573 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:47.203526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:47.204303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:47.204744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:47.205679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:47.206139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:47.206724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:47.226294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:47.243239 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b6200091-6ed5-453e-8f2d-94809526ace5 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059476 Oct 16 01:03:47.243842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 422/815] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:47 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:47.636833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:47.645040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:47.646459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:47.647894 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cbd0d02f-6c5c-4cab-8566-3937df2596e4 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.545036 Oct 16 01:03:47.648464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 394/816] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:47 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:48.205556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:48.206167 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:48.215248 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.215730 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.216057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.216774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.217423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.217855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.257688 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:48.260456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:48.272589 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.273483 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.274159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.275152 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.277124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.278401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.358016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-dbe7fc2a-a900-4a39-a5ed-ca4d107d779c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.105366 Oct 16 01:03:48.358617 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 395/817] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:48 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:48.624304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:48.635095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1aaad986-02f8-4d98-8ed5-34b2266323c0 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1517 microversion: 2.1 time: 0.433322 Oct 16 01:03:48.635992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 423/818] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:48 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1517 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:48.688137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:48.688959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:48.698133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.698821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.699445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.700347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:48.700982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:48.701575 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:48.960348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:48.970174 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:48.978839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:48.980382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4edfed66-2954-491b-9a62-463c5c1ab115 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.311041 Oct 16 01:03:48.995531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 396/819] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:48 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:49.370843 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:49.373142 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:49.379847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.380868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.381620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.382433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.383162 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.383647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.432616 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-85bbc19d-997b-4010-a4d3-88091d41003c tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.066155 Oct 16 01:03:49.433208 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 424/820] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:03:49 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments => generated 25 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:03:49.445415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:49.446644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:49.453040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.453521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.453953 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.454641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.455133 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.455551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.550110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] HTTP exception thrown: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e is not attached to volume 992410fd-c57f-46d6-8158-3244996d4182 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Request method failure captured: Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments/992410fd-c57f-46d6-8158-3244996d4182 HTTP/1.1 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1fbr0w-q7dU0P6MFGh3UkzXpnEmyCT8C83DPbOPn76Fe8cQ0LazhgR8DBUlPZPGgtZLecB2W8P-jM3R0mDFnrJh9842Z1IDlNFnczspY-7V0BRAWIW68--ErmFiTq_LF8H8-2ZaXTP4hqvg57q1gFKfc8xpyiRxnN3IUvh1qMS2uYk Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 343c019304314c669a3381cc4c68e481 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:49.559583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/343c019304314c669a3381cc4c68e481"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/343c019304314c669a3381cc4c68e481"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_343c019304314c669a3381cc4c68e481", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/343c019304314c669a3381cc4c68e481"}]}] Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 343c019304314c669a3381cc4c68e481 Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-TestMinimumBasicScenario-1525636228-project Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: a49486f43ff942bcbf4ef782d6eb6238 Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-TestMinimumBasicScenario-1525636228-project Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:49.561336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e is not attached to volume 992410fd-c57f-46d6-8158-3244996d4182 Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'server_id': 'aa091bd0-0127-4e6a-8b16-a7956cc5379e', 'id': '992410fd-c57f-46d6-8158-3244996d4182'} Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 992410fd-c57f-46d6-8158-3244996d4182. Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:49.562245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e is not attached to volume 992410fd-c57f-46d6-8158-3244996d4182 Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e is not attached to volume 992410fd-c57f-46d6-8158-3244996d4182 Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:49.563883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Returning 404 to user: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e is not attached to volume 992410fd-c57f-46d6-8158-3244996d4182 {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:49.564943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f35ddc85-af2a-431c-bd75-872425945aad tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments/992410fd-c57f-46d6-8158-3244996d4182" status: 404 len: 154 microversion: 2.1 time: 0.121294 Oct 16 01:03:49.564943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 397/821] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:03:49 2022] DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e/os-volume_attachments/992410fd-c57f-46d6-8158-3244996d4182 => generated 154 bytes in 122 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 16 01:03:49.663497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:49.664409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:49.677089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.678441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.680036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.681051 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:49.681620 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:49.682268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:49.835959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:49.845802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5c8f1adf-95db-4ebc-9837-5632f01ca468 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1519 microversion: 2.1 time: 0.189558 Oct 16 01:03:49.847204 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 425/822] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:49 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1519 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:50.010896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:50.011884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:50.018533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.019125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.019638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.020416 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.020985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.021687 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.176972 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:50.193612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:50.195073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:50.196233 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-11f93b68-d291-4963-9f5a-6a799f261e76 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.192486 Oct 16 01:03:50.196804 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 398/823] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:50 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:50.555131 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "name": "network-vif-plugged", "status": "completed", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:50.566844 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.567458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.568372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.581709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] Creating event network-vif-plugged:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:03:50.588048 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-c164b3b8-70bf-4b30-a46a-9faea8f58978 req-3797dbd4-75d7-413b-adcc-ddf7ac22f248 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037062 Oct 16 01:03:50.589036 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 426/824] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:50.740456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:50.741560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:50.748324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.748810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.749250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.749958 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.750415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.750843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.777661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:03:50.780710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:50.859142 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-173bd352-1302-43f3-8603-5876eee90bed tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 204 len: 0 microversion: 2.1 time: 0.122202 Oct 16 01:03:50.860007 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 399/825] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:03:50 2022] DELETE /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:03:50.872122 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:50.872364 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:50.873134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:50.873462 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:50.880103 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.880320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.881265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.881441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.881789 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.882204 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.882529 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.883070 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.883258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:50.883506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:50.883929 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:50.884438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:51.024088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:51.028224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:51.032136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a2e955fc-04cb-42bf-adc4-b2b1f17e83e4 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1552 microversion: 2.1 time: 0.167256 Oct 16 01:03:51.032713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 427/826] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:50 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1552 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:51.035906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d9a9b26c-508d-46d9-92a2-4a7e5d3d6025 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.168247 Oct 16 01:03:51.036477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 400/827] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:50 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:51.221675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:51.222356 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:51.228928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:51.229438 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:51.229954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:51.230651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:51.231128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:51.231619 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:51.369017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:51.377729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:51.379292 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:51.380470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-60dff984-c986-46b4-bc17-49fd5f7d785a tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.162684 Oct 16 01:03:51.381726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 428/828] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:51 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:52.055320 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:52.056036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:52.057332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:52.057960 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:52.062652 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.063173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.063621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.064295 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.064407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.064746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.064857 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.065596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.065713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.065993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.066466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.066926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.196172 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:52.203806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:52.204233 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-77c10995-94dd-41ff-987a-f8f9454df088 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1552 microversion: 2.1 time: 0.152806 Oct 16 01:03:52.204821 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 401/829] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:52 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1552 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:52.212349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ef5ddc9b-0373-4bfc-a954-064f89e06b24 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.158776 Oct 16 01:03:52.212928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 429/830] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:52 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:52.407306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:52.407830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:52.413859 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.414344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.414899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.415607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:52.416062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:52.416491 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:52.550017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:52.558220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:52.559662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:52.560808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0fa8169-ca07-4a7e-9c73-452c27a1d19c tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.157700 Oct 16 01:03:52.561365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 402/831] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:52 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:53.228696 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:53.229263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:53.236030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.236511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.237016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:53.237487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.237848 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.238160 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:53.238299 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.238924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.242837 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:03:53.246593 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.247188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.247847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.248667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.249343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.249844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.400129 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:53.408498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-90776b32-8ab7-4700-9038-b19e8d4ae07e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.176239 Oct 16 01:03:53.409397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 403/832] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:53 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:53.423054 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:53.432361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2c10b96f-5401-498b-8eb9-b398423497b9 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1552 microversion: 2.1 time: 0.208220 Oct 16 01:03:53.433023 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 430/833] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:53 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1552 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:53.585874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:53.586488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:53.594012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.594792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.595616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.596039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:53.596424 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:53.596835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:53.819533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:53.828659 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:53.833581 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:53.836038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0b8aa4f-3cf2-4b59-9d1e-d877d0d7e001 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.254303 Oct 16 01:03:53.836835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 404/834] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:53 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:54.102597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa091bd0-0127-4e6a-8b16-a7956cc5379e", "name": "network-vif-deleted", "tag": "c570709d-29cc-4c4f-a39b-9bfd4da608ed"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:03:54.113940 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.114430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.114886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.127591 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] Creating event network-vif-deleted:c570709d-29cc-4c4f-a39b-9bfd4da608ed for instance aa091bd0-0127-4e6a-8b16-a7956cc5379e on hv-cinder-maste Oct 16 01:03:54.133441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-37b73dfd-7f35-4afd-a6db-6514cc911620 req-09d5b002-4814-452d-9431-3bae60f17988 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036031 Oct 16 01:03:54.134044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 431/835] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:03:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:03:54.437060 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:54.438688 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:54.449245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.450207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.451039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.452205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.452932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.453564 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.458042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:54.459006 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:54.481458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.485374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.485888 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.486646 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.487144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.491059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.570657 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.571429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.572376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.573095 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lazy-loading 'fault' on Instance uuid aa091bd0-0127-4e6a-8b16-a7956cc5379e {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:03:54.635545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a49486f43ff942bcbf4ef782d6eb6238', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '343c019304314c669a3381cc4c68e481', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:54.643467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-bc71af2d-4b41-482b-8e16-f904ec0080b3 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 200 len: 1348 microversion: 2.1 time: 0.191425 Oct 16 01:03:54.650014 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 432/836] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:54 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 1348 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:54.657708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:54.673686 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e1bc76ad-09ac-4022-9120-79f2bd2ac5ae tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.241599 Oct 16 01:03:54.680131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 405/837] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:54 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:54.864026 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:54.864639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:54.871758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.872246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.872685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:54.873385 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:54.873849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:54.874278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:55.013115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:55.022360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:55.023847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:55.025026 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-397276d0-024c-425e-8b4c-8b48f3fbccc5 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1728 microversion: 2.20 time: 0.165075 Oct 16 01:03:55.025597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 433/838] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:54 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1728 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:03:55.673421 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:55.674888 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] [instance: aa091bd0-0127-4e6a-8b16-a7956cc5379e] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:55.681845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:55.682325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:55.682756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:55.683473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:55.683923 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:55.684340 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:55.697523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:55.698376 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:55.700906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] HTTP exception thrown: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Request method failure captured: Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e HTTP/1.1 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1fbr0w-q7dU0P6MFGh3UkzXpnEmyCT8C83DPbOPn76Fe8cQ0LazhgR8DBUlPZPGgtZLecB2W8P-jM3R0mDFnrJh9842Z1IDlNFnczspY-7V0BRAWIW68--ErmFiTq_LF8H8-2ZaXTP4hqvg57q1gFKfc8xpyiRxnN3IUvh1qMS2uYk Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 343c019304314c669a3381cc4c68e481 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:03:55.702557 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/343c019304314c669a3381cc4c68e481"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/343c019304314c669a3381cc4c68e481"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_343c019304314c669a3381cc4c68e481", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/343c019304314c669a3381cc4c68e481"}]}] Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 343c019304314c669a3381cc4c68e481 Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1525636228 Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-TestMinimumBasicScenario-1525636228-project Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: a49486f43ff942bcbf4ef782d6eb6238 Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-TestMinimumBasicScenario-1525636228-project Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. Oct 16 01:03:55.704568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'aa091bd0-0127-4e6a-8b16-a7956cc5379e'} Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:03:55.705508 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. Oct 16 01:03:55.707097 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:03:55.708381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Returning 404 to user: Instance aa091bd0-0127-4e6a-8b16-a7956cc5379e could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:03:55.708381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f7bf4f7c-2d88-4bbe-8c92-3c88dd56ec42 tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e" status: 404 len: 111 microversion: 2.1 time: 0.034980 Oct 16 01:03:55.708381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 406/839] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:55 2022] GET /compute/v2.1/servers/aa091bd0-0127-4e6a-8b16-a7956cc5379e => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:03:55.708885 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:55.709430 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:55.709962 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:55.711205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:55.711717 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:55.712217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:55.714951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6d0efd04-2d3e-42a5-9bf7-c49f6f493b5a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] Calling method '.version_select at 0x7f8bbd85a040>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:55.736050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6d0efd04-2d3e-42a5-9bf7-c49f6f493b5a tempest-TestMinimumBasicScenario-1525636228 tempest-TestMinimumBasicScenario-1525636228-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-890825791" status: 202 len: 0 microversion: 2.1 time: 0.024865 Oct 16 01:03:55.736862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 407/840] 10.222.0.25 () {58 vars in 1269 bytes} [Sun Oct 16 01:03:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-890825791 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:03:55.868091 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:55.878540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3ec6f984-d24e-4080-99c4-ca95cd790c2a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.185424 Oct 16 01:03:55.888781 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 434/841] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:55 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:03:56.054768 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:56.055657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:56.062406 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:56.063245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:56.063909 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:56.064786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:56.065251 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:56.065681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:56.219533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:56.230436 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:56.231963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:56.233088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-87401113-40eb-46e9-a751-a5a720b5d096 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1715 microversion: 2.20 time: 0.185370 Oct 16 01:03:56.233625 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 408/842] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:03:56 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1715 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:03:56.902666 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:56.903350 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:56.909812 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:56.910323 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:56.910807 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:56.911902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:56.912428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:56.912865 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:57.132102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:57.140627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-daad7f32-711e-4345-8d37-5d2ca7637614 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.242032 Oct 16 01:03:57.141303 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 435/843] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:56 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:58.167187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:58.167985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:58.174584 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:58.175123 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:58.186030 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:58.186980 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:58.187467 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:58.188311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:58.492298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:58.508462 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c9902de9-c9b0-4ed2-9e1c-1bdc148b672e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.347427 Oct 16 01:03:58.510477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 409/844] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:58 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:03:59.531496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:03:59.532117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:03:59.539788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:59.540353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:59.540862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:59.541636 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:03:59.542103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:03:59.542529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:03:59.783722 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:03:59.795032 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-cc7b6acc-2124-482c-a103-a097081b1282 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.267037 Oct 16 01:03:59.796340 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 436/845] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:03:59 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:00.820018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:00.820564 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:00.828139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:00.828613 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:00.829046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:00.829739 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:00.830368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:00.830988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:00.988120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:00.998062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-b5765319-dccf-46e0-9dc0-e53f8bb283bd tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.182342 Oct 16 01:04:00.999334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 410/846] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:00 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:02.022083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:02.022983 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:02.025716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:02.034177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:02.034675 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:02.035256 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:02.035998 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:02.039209 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:02.039732 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:02.260928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:02.269203 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1d69b390-7c7b-4566-a853-7b3d91f0603f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.251460 Oct 16 01:04:02.269877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 437/847] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:02 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:03.292377 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:03.293778 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:03.304979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:03.306265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:03.306839 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:03.310193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:03.310193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:03.310193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:03.573019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:03.582069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0f38e6c0-7b5e-4997-abb5-da1a98500cfc tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.293775 Oct 16 01:04:03.584858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 411/848] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:03 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:04.605528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:04.606315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:04.614100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:04.615564 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:04.616353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:04.617569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:04.618198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:04.618814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:04.737574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "name": "network-vif-plugged", "status": "completed", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:04.756037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:04.756868 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:04.757654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:04.771668 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] Creating event network-vif-plugged:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:04:04.778313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-4aace515-e9ba-47e3-86c1-f8fd52588479 req-24661d93-9258-45bb-b17d-38edd33aa078 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045233 Oct 16 01:04:04.779701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 412/849] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:04:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:04:04.820175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:04.829836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-01246599-b353-44e4-9e00-c353fb636a5f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.228035 Oct 16 01:04:04.832090 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 438/850] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:04 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:04:05.854573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:05.855309 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:05.861898 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:05.862393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:05.862840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:05.863568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:05.864025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:05.864475 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:06.015710 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:06.024226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-659bb730-74d6-4836-95cf-77f2adb67fbb tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1575 microversion: 2.1 time: 0.174339 Oct 16 01:04:06.024810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 413/851] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:05 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1575 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:07.047368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:07.047980 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:07.054404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:07.054951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:07.055429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:07.056172 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:07.056628 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:07.057056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:07.236563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:07.245098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-433d3aaa-4fd9-4fe9-bd08-ea83c1d752e4 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1562 microversion: 2.1 time: 0.201571 Oct 16 01:04:07.245677 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 439/852] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:07 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1562 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:08.970326 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:08.970921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:08.980235 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:08.980718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:08.981151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:08.981842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:08.982292 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:08.982711 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:09.007638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6ddb784a-7aeb-4143-bfb8-c75dd84baabd tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/ips" status: 200 len: 147 microversion: 2.1 time: 0.042388 Oct 16 01:04:09.008866 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 414/853] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:04:08 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/ips => generated 147 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:13.435996 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "18849bfb-ce16-4e05-abd5-b86c5c5e7fc5"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:13.438456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:13.445421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:13.445906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:13.446344 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:13.447328 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:13.447799 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:13.448229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:13.451929 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=100 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:15.988138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-83fc0416-c5de-4897-b40a-e8ae2499aa5c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.556009 Oct 16 01:04:15.988868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 440/854] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:04:13 2022] POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments => generated 194 bytes in 2564 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:23.689404 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:23.692093 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=23,nova_api:DELETE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:23.693660 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:23.700708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:23.701191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:23.701621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:23.922674 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Going to try to stop instance {{(pid=119013) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 16 01:04:23.965298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-225cab75-cbf1-428a-88ca-baf85539af1c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action" status: 202 len: 0 microversion: 2.1 time: 0.279654 Oct 16 01:04:23.965907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 415/855] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:04:23 2022] POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action => generated 0 bytes in 281 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:04:23.973153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=7 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:23.978405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:23.979525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:23.986647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:23.987188 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:23.987625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:23.988322 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:23.988776 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:23.989223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:24.128246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:24.136698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b900ae7c-a077-4ec9-b6b0-73cecae7f1fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.165761 Oct 16 01:04:24.137269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 441/856] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:23 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:25.161226 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:25.162342 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:25.169476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:25.169993 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:25.170426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:25.171153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:25.171601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:25.172023 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:25.308737 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:25.316959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7bc07d15-83d0-491d-bbbb-f50248f8a839 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.160711 Oct 16 01:04:25.317526 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 416/857] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:25 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:26.340415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:26.341039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:26.347756 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:26.348252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:26.348696 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:26.349445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:26.349941 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:26.350369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:26.483839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:26.492762 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1d148754-17de-4207-b1fe-50e59c87065e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.156322 Oct 16 01:04:26.493331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 442/858] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:26 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:27.516342 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:27.516957 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:27.526592 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:27.527116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:27.527563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:27.528281 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:27.528736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:27.529165 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:27.665067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:27.673558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-475ff4de-2875-4ea4-81c7-f40861ab0722 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.160957 Oct 16 01:04:27.674901 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 417/859] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:27 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:28.697458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:28.698100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:28.704558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:28.705034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:28.705474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:28.706168 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:28.706625 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:28.707087 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:28.858712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:28.867134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f534c49a-ad55-4d1d-b36d-22ece13c25f6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.173884 Oct 16 01:04:28.867715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 443/860] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:28 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:29.889754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:29.890454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:29.897527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:29.898253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:29.898850 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:29.899716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:29.900326 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:29.900810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:30.048385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:30.057978 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-471acfe1-5328-42f4-9a2f-ae9e81b531c2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.172241 Oct 16 01:04:30.058562 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 418/861] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:29 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:31.081661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:31.082310 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:31.088784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:31.089269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:31.089705 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:31.090401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:31.090912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:31.091349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:31.222144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:31.231084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e53a2ea0-4770-4b53-b125-b4063850d552 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.153330 Oct 16 01:04:31.231650 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 444/862] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:31 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:32.254757 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:32.256159 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:32.263472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:32.264070 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:32.264511 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:32.265220 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:32.265681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:32.266109 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:32.403445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:32.413150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-53ab29e7-ac9d-4831-a713-00b393be1ca9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.162142 Oct 16 01:04:32.413770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 419/863] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:32 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:33.437279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:33.438150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:33.445283 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:33.445772 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:33.446209 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:33.447270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:33.447727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:33.448158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:33.593467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:33.602550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c02dde14-a69e-4ca9-af59-a0ce5b24cec3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.169064 Oct 16 01:04:33.603170 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 445/864] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:33 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:34.627219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:34.627767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:34.634989 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:34.635481 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:34.635925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:34.636626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:34.637088 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:34.637523 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:34.770890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:34.779098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8811e352-c063-437e-96f0-1099dfe2e90a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.155916 Oct 16 01:04:34.779656 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 420/865] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:34 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:35.801710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:35.802797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:35.809415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:35.809944 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:35.810453 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:35.811200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:35.811660 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:35.812085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:35.965262 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:35.973984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ea3db397-2ab3-4829-9a0b-f60427243dd5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.176037 Oct 16 01:04:35.974826 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 446/866] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:35 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:36.203525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:36.205786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:36.212925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.213423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.213861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.214571 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.215069 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.215498 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.246286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-82e911f6-9da7-43fd-bfa1-37a48f94919d tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047001 Oct 16 01:04:36.246880 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 421/867] 10.222.0.25 () {60 vars in 1343 bytes} [Sun Oct 16 01:04:36 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:36.314322 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:36.314827 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:36.321704 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.322202 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.322646 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.323425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.324110 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.324548 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.351768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:04:36.433750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5a3e2d97-8dbc-4538-9464-04c411be9c18 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 204 len: 0 microversion: 2.20 time: 0.123123 Oct 16 01:04:36.434542 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 447/868] 10.222.0.25 () {60 vars in 1280 bytes} [Sun Oct 16 01:04:36 2022] DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 16 01:04:36.443959 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:36.444787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:36.451810 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.452307 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.452747 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.453459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:36.453970 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:36.454349 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:36.587031 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:36.595238 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:36.596722 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:36.597882 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8eee482d-0e7d-4666-85f7-496b79cb8c01 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.158956 Oct 16 01:04:36.598983 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 422/869] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:36 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 16 01:04:36.999354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:37.000057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:37.006580 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:37.007124 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:37.007540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:37.008233 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:37.008688 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:37.009111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:37.143148 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:37.152231 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-58e6ce9d-99a2-4627-bb0b-da5e978b7d05 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.157151 Oct 16 01:04:37.152823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 448/870] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:36 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:37.625455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:37.626143 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:37.633509 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:37.634012 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:37.634455 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:37.635292 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:37.636059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:37.636223 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:37.781084 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:37.793076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:37.794521 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:37.795714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-992ada2d-4a5d-497b-aae1-2e3b0273aec0 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1721 microversion: 2.20 time: 0.174262 Oct 16 01:04:37.796269 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 423/871] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:37 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:04:38.176178 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:38.177489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:38.184878 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:38.185373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:38.185804 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:38.186498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:38.186984 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:38.187407 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:38.320072 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:38.328641 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8cd6651b-0923-48bc-8f25-e9859fd29c0f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1725 microversion: 2.1 time: 0.156295 Oct 16 01:04:38.329221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 449/872] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:38 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:38.821583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:38.822818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:38.830033 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:38.830749 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:38.831374 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:38.832287 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:38.832916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:38.833528 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:38.862263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0f07f47f-8851-4e67-9c77-d4115ff157c7", "name": "network-vif-deleted", "tag": "544a970e-fd66-4b0a-ba39-0dcfeed391f8"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:38.880545 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:38.881044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:38.881465 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:38.924285 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] Creating event network-vif-deleted:544a970e-fd66-4b0a-ba39-0dcfeed391f8 for instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 on hv-cinder-maste Oct 16 01:04:38.932286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-1a4f3e52-8b7b-451a-828c-7113f87b8b6c req-7a4632af-5fde-41ea-8446-97a11950b603 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077906 Oct 16 01:04:38.933668 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 450/873] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:04:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:04:38.985275 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:38.995020 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:39.002467 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ccece85a85fb48c4887b52c96c5bc642', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cca355ef4a294ecd88cc299b7e0ca9d8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:39.003740 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6642a41a-d030-4711-9e14-caf3596f4587 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 200 len: 1653 microversion: 2.20 time: 0.186895 Oct 16 01:04:39.004504 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 424/874] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:38 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 1653 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 16 01:04:39.353598 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:39.354316 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:39.360508 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:39.361235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:39.361679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:39.362386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:39.362841 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:39.363466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:39.511466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:39.521395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9da09097-364a-46a1-a3fb-1f32761c5c4a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1702 microversion: 2.1 time: 0.172438 Oct 16 01:04:39.522251 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 451/875] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:39 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1702 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:39.547461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:39.559488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:39.560078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:39.560585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:39.586487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Going to try to start instance {{(pid=119013) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 16 01:04:39.635856 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5e2e5a13-2403-4e00-a0f0-e1ea27328e02 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action" status: 202 len: 0 microversion: 2.1 time: 0.092505 Oct 16 01:04:39.636770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 425/876] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:04:39 2022] POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action => generated 0 bytes in 94 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:04:39.646343 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:39.647994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:39.654778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:39.655495 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:39.656022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:39.656887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:39.658118 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:39.658555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:39.817755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:39.826694 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0a046647-cd1d-4fa8-832e-3cadb3b2afdf tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1711 microversion: 2.1 time: 0.185785 Oct 16 01:04:39.827822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 452/877] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:39 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1711 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:40.035072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:40.036513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:40.045013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:40.045645 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:40.046095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:40.046987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:40.047664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:40.047984 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:40.066207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 HTTP/1.1 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:40.066507 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:40.068578 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '0f07f47f-8851-4e67-9c77-d4115ff157c7'} Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:40.070190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:40.072441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:40.075474 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:40.075474 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a8509b35-81fc-479e-ac98-78f9b728144b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 404 len: 111 microversion: 2.20 time: 0.038641 Oct 16 01:04:40.075474 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 426/878] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:40 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 16 01:04:40.077681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fb20c576-6c25-48c8-bbe1-ed1b44329203 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '.version_select at 0x7f8bbd9d60d0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:40.100215 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fb20c576-6c25-48c8-bbe1-ed1b44329203 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-532904353" status: 204 len: 0 microversion: 2.20 time: 0.026104 Oct 16 01:04:40.101227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 453/879] 10.222.0.25 () {60 vars in 1259 bytes} [Sun Oct 16 01:04:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-532904353 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 16 01:04:40.852441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:40.853347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:40.859132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:40.859621 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:40.860055 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:40.860756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:40.861205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:40.861629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:41.028397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:41.039533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-86c5d130-4eba-40c6-86c8-7cbf8d2ddfc1 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1711 microversion: 2.1 time: 0.191817 Oct 16 01:04:41.040786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 427/880] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:40 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1711 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:42.066973 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:42.067682 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:42.074574 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.075100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.075553 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.076263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.076720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.077366 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.117090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:42.118952 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:42.133412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.134269 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.135302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.136532 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.137343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.138206 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.158725 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 HTTP/1.1 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:42.161802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.163348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '0f07f47f-8851-4e67-9c77-d4115ff157c7'} Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:42.164260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.165525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.167537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:42.173833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1f84e451-9f39-4e66-ac71-e7ab76adca5e tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 404 len: 111 microversion: 2.20 time: 0.063435 Oct 16 01:04:42.180265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 428/881] 10.222.0.25 () {60 vars in 1280 bytes} [Sun Oct 16 01:04:42 2022] DELETE /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:04:42.191172 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:42.192138 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: 0f07f47f-8851-4e67-9c77-d4115ff157c7] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:42.201505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.201985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.202451 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.203529 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:42.204201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:42.204888 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:42.227213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 HTTP/1.1 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:42.229415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.231074 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '0f07f47f-8851-4e67-9c77-d4115ff157c7'} Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:42.231851 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. Oct 16 01:04:42.233635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:42.235140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance 0f07f47f-8851-4e67-9c77-d4115ff157c7 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:42.235140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e546bab8-e451-4152-84ab-10e7544ef995 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7" status: 404 len: 111 microversion: 2.20 time: 0.050037 Oct 16 01:04:42.235140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 429/882] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:42 2022] GET /compute/v2.1/servers/0f07f47f-8851-4e67-9c77-d4115ff157c7 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:04:42.250025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:04:42.259296 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e1e50faf-fdb5-4b8c-8a10-6b1c0299a0d0 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1715 microversion: 2.1 time: 0.196230 Oct 16 01:04:42.260039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 454/883] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:04:42 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1715 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:04:43.392792 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:43.393601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:43.403119 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:43.403610 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:43.404187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:43.405139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:43.405852 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:43.406576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:43.423243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb HTTP/1.1 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:43.424990 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.426977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'd07b6833-a39f-462d-9c7e-48dda346d8cb'} Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:43.427838 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.429183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.430683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:43.430683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cee1d880-f49c-44cf-b9d4-ccd34706c8c6 tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 404 len: 111 microversion: 2.20 time: 0.038088 Oct 16 01:04:43.430683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 430/884] 10.222.0.25 () {60 vars in 1280 bytes} [Sun Oct 16 01:04:43 2022] DELETE /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:04:43.439365 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:43.440077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] [instance: d07b6833-a39f-462d-9c7e-48dda346d8cb] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:04:43.448547 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:43.449194 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:43.449648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:43.450445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:04:43.451107 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:04:43.452114 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:04:43.467512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:43.469890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] HTTP exception thrown: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Request method failure captured: Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb HTTP/1.1 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1cCme6fxJ0DXuxwGafLNV10MgysBIevurNJCwdrXXULqIOIZH7acEFBhKCVekdF1MCYWZgRZ8PAOwcbTBeV7s2nYxY5X4jVPkrmpE3Hjf5ytHiIernpkDSYdD349SkUDUQFIUEcVZX54nayS17_OAUtmdbeuCvYS-iHuOTj1dS9EDg Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Openstack-Nova-Api-Version: 2.20 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:04:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cca355ef4a294ecd88cc299b7e0ca9d8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cca355ef4a294ecd88cc299b7e0ca9d8", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cca355ef4a294ecd88cc299b7e0ca9d8"}]}] Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: cca355ef4a294ecd88cc299b7e0ca9d8 Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1148799462 Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: ccece85a85fb48c4887b52c96c5bc642 Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1148799462-project Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.474298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'd07b6833-a39f-462d-9c7e-48dda346d8cb'} Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:43.475277 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. Oct 16 01:04:43.478497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:43.480448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] Returning 404 to user: Instance d07b6833-a39f-462d-9c7e-48dda346d8cb could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:43.480448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7398b8cd-3d6e-439f-876d-a8a821cc0d3b tempest-AttachVolumeShelveTestJSON-1148799462 tempest-AttachVolumeShelveTestJSON-1148799462-project] 10.222.0.25 "GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb" status: 404 len: 111 microversion: 2.20 time: 0.040465 Oct 16 01:04:43.480448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 455/885] 10.222.0.25 () {60 vars in 1277 bytes} [Sun Oct 16 01:04:43 2022] GET /compute/v2.1/servers/d07b6833-a39f-462d-9c7e-48dda346d8cb => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 16 01:04:48.160286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f474903a-619a-4751-8de5-a90ddda6e02a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Action: 'create', calling method: .version_select at 0x7f8bbd84c280>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-2142703938"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:48.452899 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f474903a-619a-4751-8de5-a90ddda6e02a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.411815 Oct 16 01:04:48.453627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 431/886] 10.222.0.25 () {60 vars in 1155 bytes} [Sun Oct 16 01:04:48 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 413 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Oct 16 01:04:48.466419 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d2514092-2d0f-41d6-8b69-7ffee4524b1e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd8e2a60>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:48.517021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d2514092-2d0f-41d6-8b69-7ffee4524b1e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 200 len: 345 microversion: 2.1 time: 0.054480 Oct 16 01:04:48.517595 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 456/887] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:04:48 2022] GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 345 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:49.532453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3a37c94f-bd2b-48d0-8b2c-cf78785e34a6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd926f70>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:49.580396 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3a37c94f-bd2b-48d0-8b2c-cf78785e34a6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 200 len: 346 microversion: 2.1 time: 0.052227 Oct 16 01:04:49.580961 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 432/888] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:04:49 2022] GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 346 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:49.595073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-de8694e9-17ae-4d01-a10b-f99d14409984 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Action: 'create', calling method: .version_select at 0x7f8bbd8ad0d0>, body: {"snapshot": {"volume_id": "a14bb2ee-2bc2-412b-bd93-b67f680bb58d", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-2008371502"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:04:49.746773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-de8694e9-17ae-4d01-a10b-f99d14409984 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.155895 Oct 16 01:04:49.752538 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 457/889] 10.222.0.25 () {60 vars in 1162 bytes} [Sun Oct 16 01:04:49 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:49.759189 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-66536dfb-8fdd-4b49-91fb-f5d77a0d7129 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd97f3a0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:49.790549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-66536dfb-8fdd-4b49-91fb-f5d77a0d7129 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 287 microversion: 2.1 time: 0.035153 Oct 16 01:04:49.791205 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 433/890] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:49 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 287 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:49.802550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-350eb81e-e9c1-4122-86f1-9f0f41334649 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbda37ca0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:49.835601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-350eb81e-e9c1-4122-86f1-9f0f41334649 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.036788 Oct 16 01:04:49.836225 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 458/891] 10.222.0.25 () {58 vars in 1140 bytes} [Sun Oct 16 01:04:49 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:04:49.847648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8a77cfc3-1749-4831-9ac4-387faa12f822 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd97f8b0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:49.877977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8a77cfc3-1749-4831-9ac4-387faa12f822 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 287 microversion: 2.1 time: 0.034063 Oct 16 01:04:49.878532 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 434/892] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:49 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:50.892211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1f1a31d4-5af2-4eb8-aa9a-01d8b0939ebc tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd9d6af0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:50.924418 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1f1a31d4-5af2-4eb8-aa9a-01d8b0939ebc tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 287 microversion: 2.1 time: 0.036690 Oct 16 01:04:50.925034 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 459/893] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:50 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:51.938124 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2a57531f-9c98-4a64-9177-a1d9d7e5d36b tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd84cd30>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:51.968997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2a57531f-9c98-4a64-9177-a1d9d7e5d36b tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 287 microversion: 2.1 time: 0.035218 Oct 16 01:04:51.969576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 435/894] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:51 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:52.982160 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0972330d-0752-4b22-a258-4cd8022b89b6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbdadf4c0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:53.012451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0972330d-0752-4b22-a258-4cd8022b89b6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 288 microversion: 2.1 time: 0.034659 Oct 16 01:04:53.013033 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 460/895] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:52 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 288 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:53.025444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-82088dc9-87c1-48f5-980c-2fc20eb31f5a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd97f8b0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:53.082607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-82088dc9-87c1-48f5-980c-2fc20eb31f5a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "DELETE /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 202 len: 0 microversion: 2.1 time: 0.061325 Oct 16 01:04:53.083206 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 436/896] 10.222.0.25 () {58 vars in 1254 bytes} [Sun Oct 16 01:04:53 2022] DELETE /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:04:53.091421 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e4a829e4-a1a6-4a5c-98d4-ea780812dd50 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd9e6160>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:53.122189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e4a829e4-a1a6-4a5c-98d4-ea780812dd50 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 200 len: 287 microversion: 2.1 time: 0.034453 Oct 16 01:04:53.122758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 461/897] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:53 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 287 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:54.132900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=32 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:54.133679 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:54.139022 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-83a8bc0a-82df-4686-b224-07184df8961e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd8d3f70>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:54.164643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-83a8bc0a-82df-4686-b224-07184df8961e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] HTTP exception thrown: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-83a8bc0a-82df-4686-b224-07184df8961e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Request method failure captured: Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce HTTP/1.1 Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1ivF0nHdYVPttaFd9J30Y5xD1IvKW2FIzZ3fbYmfy1r3sRngJPSbbnlzxltVWRejJI2y9o3Dsu4e2W2G721wizkYPHMjX4ftgu_HoX4WxVIdHpFxYSRNmmW_m02EuIMigp5HdAd5hsUtC1etyrp1hBxVUHE6grIZn8KzbDr6lMXrOg Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:54.167368 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_8e285686fd0f4ef883fbbab3a24ace5f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/8e285686fd0f4ef883fbbab3a24ace5f"}]}] Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 053c94205d4d497cafeb5c83c591b14e Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '4ced7134-e6d5-486e-8bec-1e79122284ce'} Oct 16 01:04:54.169195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. (HTTP 404) (Request-ID: req-b9828286-ff19-47dd-a2c9-fc0a3a7bab46) Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.170289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:54.171843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 16 01:04:54.173339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-83a8bc0a-82df-4686-b224-07184df8961e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Returning 404 to user: Snapshot 4ced7134-e6d5-486e-8bec-1e79122284ce could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-83a8bc0a-82df-4686-b224-07184df8961e tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce" status: 404 len: 111 microversion: 2.1 time: 0.036795 Oct 16 01:04:54.174935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 437/898] 10.222.0.25 () {58 vars in 1251 bytes} [Sun Oct 16 01:04:54 2022] GET /compute/v2.1/os-snapshots/4ced7134-e6d5-486e-8bec-1e79122284ce => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:04:54.176612 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:04:54.179789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-86d369f8-9536-4ce6-bcc3-8c67ac3a331a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbdddf940>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:54.245670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-86d369f8-9536-4ce6-bcc3-8c67ac3a331a tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 202 len: 0 microversion: 2.1 time: 0.071678 Oct 16 01:04:54.246426 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 462/899] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:04:54 2022] DELETE /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 0 bytes in 73 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:04:54.255586 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-04f581c9-e50e-4852-a89c-0d8af7b872b6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd97f8b0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:54.304076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-04f581c9-e50e-4852-a89c-0d8af7b872b6 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 200 len: 345 microversion: 2.1 time: 0.052821 Oct 16 01:04:54.304643 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 438/900] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:04:54 2022] GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 345 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:04:55.319795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3a382f79-8195-4ce5-8a43-3997892ead05 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd8e2040>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:55.348605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-3a382f79-8195-4ce5-8a43-3997892ead05 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] HTTP exception thrown: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3a382f79-8195-4ce5-8a43-3997892ead05 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Request method failure captured: Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d HTTP/1.1 Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1ivF0nHdYVPttaFd9J30Y5xD1IvKW2FIzZ3fbYmfy1r3sRngJPSbbnlzxltVWRejJI2y9o3Dsu4e2W2G721wizkYPHMjX4ftgu_HoX4WxVIdHpFxYSRNmmW_m02EuIMigp5HdAd5hsUtC1etyrp1hBxVUHE6grIZn8KzbDr6lMXrOg Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:04:55.351431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_8e285686fd0f4ef883fbbab3a24ace5f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/8e285686fd0f4ef883fbbab3a24ace5f"}]}] Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 053c94205d4d497cafeb5c83c591b14e Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'a14bb2ee-2bc2-412b-bd93-b67f680bb58d'} Oct 16 01:04:55.353075 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. (HTTP 404) (Request-ID: req-3add08a2-5c16-4700-aa4c-41b5f720207d) Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.354241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:55.355805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 16 01:04:55.357244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3a382f79-8195-4ce5-8a43-3997892ead05 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Returning 404 to user: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3a382f79-8195-4ce5-8a43-3997892ead05 tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 404 len: 109 microversion: 2.1 time: 0.037700 Oct 16 01:04:55.358655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 463/901] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:04:55 2022] GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:04:55.367885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1e2177ef-db05-4ba6-b304-709ccce2136d tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbd926940>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:55.395433 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-1e2177ef-db05-4ba6-b304-709ccce2136d tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] HTTP exception thrown: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1e2177ef-db05-4ba6-b304-709ccce2136d tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Request method failure captured: Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d HTTP/1.1 Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1ivF0nHdYVPttaFd9J30Y5xD1IvKW2FIzZ3fbYmfy1r3sRngJPSbbnlzxltVWRejJI2y9o3Dsu4e2W2G721wizkYPHMjX4ftgu_HoX4WxVIdHpFxYSRNmmW_m02EuIMigp5HdAd5hsUtC1etyrp1hBxVUHE6grIZn8KzbDr6lMXrOg Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:04:55.397817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_8e285686fd0f4ef883fbbab3a24ace5f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/8e285686fd0f4ef883fbbab3a24ace5f"}]}] Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 053c94205d4d497cafeb5c83c591b14e Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'a14bb2ee-2bc2-412b-bd93-b67f680bb58d'} Oct 16 01:04:55.399390 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. (HTTP 404) (Request-ID: req-8145c86c-4b23-41d3-b2f8-040dc0349809) Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.400246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 16 01:04:55.401520 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 16 01:04:55.402795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-1e2177ef-db05-4ba6-b304-709ccce2136d tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Returning 404 to user: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-1e2177ef-db05-4ba6-b304-709ccce2136d tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 404 len: 109 microversion: 2.1 time: 0.036459 Oct 16 01:04:55.404014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 439/902] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:04:55 2022] DELETE /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:04:55.408102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9aaa45c8-030c-4df9-a2d9-9746f8ab30de tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Calling method '.version_select at 0x7f8bbe8e1ee0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:04:55.436755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-9aaa45c8-030c-4df9-a2d9-9746f8ab30de tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] HTTP exception thrown: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9aaa45c8-030c-4df9-a2d9-9746f8ab30de tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Request method failure captured: Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d HTTP/1.1 Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1ivF0nHdYVPttaFd9J30Y5xD1IvKW2FIzZ3fbYmfy1r3sRngJPSbbnlzxltVWRejJI2y9o3Dsu4e2W2G721wizkYPHMjX4ftgu_HoX4WxVIdHpFxYSRNmmW_m02EuIMigp5HdAd5hsUtC1etyrp1hBxVUHE6grIZn8KzbDr6lMXrOg Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:04:55.438803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/8e285686fd0f4ef883fbbab3a24ace5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_8e285686fd0f4ef883fbbab3a24ace5f", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/8e285686fd0f4ef883fbbab3a24ace5f"}]}] Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: 8e285686fd0f4ef883fbbab3a24ace5f Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-700462592 Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 053c94205d4d497cafeb5c83c591b14e Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesSnapshotsTestJSON-700462592-project Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'a14bb2ee-2bc2-412b-bd93-b67f680bb58d'} Oct 16 01:04:55.440724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. (HTTP 404) (Request-ID: req-e395aa16-81fd-43d5-9928-f8d5b5db3f86) Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.441617 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 16 01:04:55.443158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 16 01:04:55.444511 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9aaa45c8-030c-4df9-a2d9-9746f8ab30de tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] Returning 404 to user: Volume a14bb2ee-2bc2-412b-bd93-b67f680bb58d could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9aaa45c8-030c-4df9-a2d9-9746f8ab30de tempest-VolumesSnapshotsTestJSON-700462592 tempest-VolumesSnapshotsTestJSON-700462592-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d" status: 404 len: 109 microversion: 2.1 time: 0.036160 Oct 16 01:04:55.445908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 464/903] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:04:55 2022] GET /compute/v2.1/os-volumes/a14bb2ee-2bc2-412b-bd93-b67f680bb58d => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:05:23.926874 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:23.927486 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:23.934423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:23.935552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:23.935552 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:23.936386 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:23.936862 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:23.937917 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:24.029489 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lazy-loading 'services' on Instance uuid f3182305-f754-4b0f-989d-7362967cb10a {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:05:24.106036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9ef72d35-b7f0-453f-b40f-0580d34caef8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments/18849bfb-ce16-4e05-abd5-b86c5c5e7fc5" status: 202 len: 0 microversion: 2.1 time: 0.183692 Oct 16 01:05:24.106657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 440/904] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:05:23 2022] DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments/18849bfb-ce16-4e05-abd5-b86c5c5e7fc5 => generated 0 bytes in 185 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:05:25.236008 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:25.240829 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:25.247965 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:25.248458 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:25.248908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:25.310537 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Going to try to stop instance {{(pid=119012) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 16 01:05:25.353749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-421595ae-9b5d-47c2-bb22-0e2469565a0b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action" status: 202 len: 0 microversion: 2.1 time: 0.118880 Oct 16 01:05:25.354342 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 465/905] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:05:25 2022] POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:05:25.363649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:25.364344 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:25.371194 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:25.371681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:25.372118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:25.372814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:25.373265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:25.373688 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:25.503478 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:25.511441 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5e1ec230-0715-422e-9cc2-e471ea35bc73 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.152513 Oct 16 01:05:25.512013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 441/906] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:25 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:26.536709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:26.537332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:26.544576 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:26.545070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:26.545571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:26.546314 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:26.546811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:26.547433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:26.695263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:26.704102 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-dd7039c9-90af-48cf-b51a-e121c9556e17 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.172187 Oct 16 01:05:26.705218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 466/907] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:26 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:27.729214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:27.729866 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:27.736572 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:27.737064 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:27.737505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:27.738215 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:27.738678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:27.739139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:27.879573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:27.888180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-40cb95d6-68e1-49c1-84f4-e1adec9bd64a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.163039 Oct 16 01:05:27.888764 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 442/908] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:27 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:28.902907 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:28.919225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:28.919709 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:28.920427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:28.934329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] Creating event network-changed:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:05:28.940038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-08938fa7-6690-4733-b1e0-b19d3909fadc req-966ba456-b24f-4951-9087-5e6816744ebd service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040709 Oct 16 01:05:28.942648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 443/909] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:05:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:05:28.948539 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:28.949227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:28.955753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:28.956241 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:28.956679 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:28.957380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:28.957836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:28.958261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:29.098639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:29.107019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e0bcdc33-ebc6-4b60-937e-0192eaa972e4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.162817 Oct 16 01:05:29.115095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 444/910] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:28 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:29.134329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:29.134903 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:29.140789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:29.141267 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:29.141698 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:29.142393 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:29.142844 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:29.143336 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:29.169136 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:05:29.251247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-df7fbe95-b0c6-4f2e-8de3-4fd9eccafe9d tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "DELETE /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 204 len: 0 microversion: 2.1 time: 0.353496 Oct 16 01:05:29.251842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 467/911] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:05:28 2022] DELETE /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:05:29.260624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:29.261372 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:29.267714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:29.268286 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:29.268963 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:29.269751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:29.270306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:29.275014 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:29.408585 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:29.417654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a01e255-f6aa-46a0-913d-45866ac2724c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.161200 Oct 16 01:05:29.418233 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 445/912] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:29 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:30.132711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:30.133423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:30.140128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:30.140957 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:30.141077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:30.141715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:30.142174 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:30.142594 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:30.293859 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:30.302257 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3c024c6a-9de3-4786-981e-66b57de70445 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.174462 Oct 16 01:05:30.302838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 468/913] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:30 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:30.441162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:30.441872 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:30.449574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:30.450068 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:30.450513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:30.451244 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:30.451701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:30.452151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:30.595442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:30.603808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2cd2caa7-bad7-4038-8ed2-233d05717d68 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.166591 Oct 16 01:05:30.604370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 446/914] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:30 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:31.326383 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:31.327069 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:31.333700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:31.334220 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:31.334659 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:31.336060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:31.336524 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:31.336954 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:31.471802 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:31.480016 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2d5ce197-5a2d-43eb-b6d4-273f44d5de4c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.157551 Oct 16 01:05:31.480594 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 469/915] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:31 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:31.627501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:31.628234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:31.635635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:31.636128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:31.636570 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:31.637272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:31.637733 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:31.638167 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:31.979730 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:31.987884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2827f9ec-234a-439c-be32-d3e7f44e9069 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1568 microversion: 2.1 time: 0.364176 Oct 16 01:05:31.988666 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 447/916] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:31 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1568 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:32.339902 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f92209b-6feb-4e65-af10-6872f24bbc95", "name": "network-vif-deleted", "tag": "a16383c6-9bce-4108-ad57-fad77bb41e2b"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:32.351353 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:32.352068 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:32.352571 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:32.364413 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] Creating event network-vif-deleted:a16383c6-9bce-4108-ad57-fad77bb41e2b for instance 2f92209b-6feb-4e65-af10-6872f24bbc95 on hv-cinder-maste Oct 16 01:05:32.370278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-3d0237c2-4365-4c07-b19d-0df0b7d5b8d8 req-85d9076c-7dea-4d64-9411-94214b50f0f6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034973 Oct 16 01:05:32.370908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 470/917] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:05:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:05:32.504858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:32.505408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:32.511842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:32.512333 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:32.512775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:32.513487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:32.513966 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:32.514378 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:32.648801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:32.657121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a396ad99-36bd-454a-97fc-6574fc0402ee tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.156276 Oct 16 01:05:32.657825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 448/918] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:32 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:33.013919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:33.014540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:33.021029 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:33.021517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:33.021955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:33.023208 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:33.023760 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:33.024192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:33.089834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:33.098039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c10cba95-2e85-4cb3-81fd-272c8427fb16 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1220 microversion: 2.1 time: 0.088431 Oct 16 01:05:33.098615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 471/919] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:33 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:33.682662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:33.683291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:33.689905 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:33.690401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:33.690842 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:33.691576 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:33.692038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:33.692475 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:33.834607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:33.843065 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-50df5218-6c5c-4194-870e-b2fcbad6df99 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.165367 Oct 16 01:05:33.843652 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 449/920] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:33 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:34.121842 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:34.122521 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:34.129689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:34.130201 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:34.131109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:34.131372 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:34.132373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:34.132782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:34.208431 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:34.218211 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7ffa48c6-d3fe-4be6-bf07-3bcdacbed288 tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1220 microversion: 2.1 time: 0.100450 Oct 16 01:05:34.218806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 472/921] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:34 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1220 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:34.868486 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:34.869081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:34.875771 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:34.876260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:34.876699 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:34.877415 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:34.877873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:34.878369 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:35.011382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:35.019718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5889e4c4-10da-4ebc-a12c-8a00a717f471 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.156018 Oct 16 01:05:35.020311 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 450/922] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:34 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:35.240325 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:35.240967 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:35.247908 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:35.248394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:35.248835 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:35.249535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:35.249989 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:35.250414 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:35.314750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:35.323111 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-53d0e7ef-4088-47e5-9913-f6d529ae938a tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1220 microversion: 2.1 time: 0.086575 Oct 16 01:05:35.323741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 473/923] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:35 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:36.043770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:36.044487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:36.050991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:36.051569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:36.052077 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:36.052845 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:36.053372 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:36.054510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:36.193885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:36.203657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-45319c0c-6f4b-4b94-bda1-55a7ce6ca548 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.163646 Oct 16 01:05:36.204967 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 451/924] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:36 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:36.346246 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:36.346883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:36.353931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:36.354425 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:36.354894 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:36.355586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:36.356056 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:36.356487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:36.419550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:36.429181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0f08fb04-a549-4872-9a09-90228cc9113e tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1220 microversion: 2.1 time: 0.086080 Oct 16 01:05:36.429975 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 474/925] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:36 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:37.245427 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:37.246120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:37.253289 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.254054 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.255046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.255824 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.256418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.257460 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.375038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-393239365", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-466177183", "security_groups": [{"name": "tempest-secgroup-smoke-1119436186"}], "block_device_mapping_v2": [{"uuid": "dbc79073-775a-4724-a883-d813eaff4ad1", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "67400858-429f-4052-a9a5-3012690406d5"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:37.420911 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:37.430875 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-8083fb79-09cc-4c26-87f2-c362aab2b681 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.206534 Oct 16 01:05:37.443147 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 452/926] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:37 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:37.453049 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:37.453955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:37.460502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.461227 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.461889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.462587 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.463107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.463533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.497182 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.497904 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.498336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.498793 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lazy-loading 'fault' on Instance uuid 2f92209b-6feb-4e65-af10-6872f24bbc95 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:05:37.530845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] validate_networks() for [('67400858-429f-4052-a9a5-3012690406d5', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:05:37.544326 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b899e194369a4efcb87d7cb92dc5f902', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed24cab62e1f4810a10e7ba36fcbeadb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:37.555141 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-904d1be9-4852-4641-b6ee-9eb5b9c8feee tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 200 len: 1225 microversion: 2.1 time: 0.105477 Oct 16 01:05:37.572607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 453/927] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:37 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:05:37.952632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:05:37.953071 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:05:37.953550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:05:37.953972 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:05:37.954386 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:05:37.963484 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.964067 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.964412 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.965379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:37.965769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:37.966373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:37.986525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:05:37.991744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Getting quotas for user c4e65cef89af48a7a51e5548634de75f and project cb1743385ea14cc39244c83e8fc88e7e. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:05:38.051523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:05:38.054294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='dbc79073-775a-4724-a883-d813eaff4ad1',volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:05:38.184622 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3f0fb106-f8d3-4b5c-a41f-bb533789426c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.813376 Oct 16 01:05:38.185455 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 475/928] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:05:37 2022] POST /compute/v2.1/servers => generated 406 bytes in 815 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 16 01:05:38.198116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-92769d87-27b4-4a52-a5e8-6d50feeb2663 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:38.198743 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-92769d87-27b4-4a52-a5e8-6d50feeb2663 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:38.250328 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-92769d87-27b4-4a52-a5e8-6d50feeb2663 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:38.258302 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-92769d87-27b4-4a52-a5e8-6d50feeb2663 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1093 microversion: 2.1 time: 0.065721 Oct 16 01:05:38.258888 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 454/929] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:38 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1093 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:38.458155 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:38.458687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:38.465192 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:38.466289 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:38.466815 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:38.467664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:38.468128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:38.468566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:38.579930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:38.580548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] [instance: 2f92209b-6feb-4e65-af10-6872f24bbc95] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:38.586681 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:38.587218 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:38.587663 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:38.588378 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:38.588836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:38.589265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:38.603461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] HTTP exception thrown: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. Oct 16 01:05:38.604836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Request method failure captured: Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 HTTP/1.1 Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1ej5NN3X7e8IY7puGzcdoFPCXJnww73nWy-wn17M_LE0Yro0gJE5Sgv-onu6RZHiMKiCewQsMSogRQL1yGYBA8Avwdkcjl0nRpcJCcdOb9i5Kaa1UFid_ZMlukneYSl0qThTyR1UhI9vnK1L8qhzU6cry_Xfvr8BwxdQt-uFy-keKw Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: ed24cab62e1f4810a10e7ba36fcbeadb Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-TestShelveInstance-628145774 Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:05:38.605112 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ed24cab62e1f4810a10e7ba36fcbeadb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/ed24cab62e1f4810a10e7ba36fcbeadb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_ed24cab62e1f4810a10e7ba36fcbeadb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/ed24cab62e1f4810a10e7ba36fcbeadb"}]}] Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-TestShelveInstance-628145774 Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: ed24cab62e1f4810a10e7ba36fcbeadb Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-TestShelveInstance-628145774 Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-TestShelveInstance-628145774-project Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: b899e194369a4efcb87d7cb92dc5f902 Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-TestShelveInstance-628145774-project Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. Oct 16 01:05:38.606890 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '2f92209b-6feb-4e65-af10-6872f24bbc95'} Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:05:38.607976 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. Oct 16 01:05:38.609423 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:05:38.610799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Returning 404 to user: Instance 2f92209b-6feb-4e65-af10-6872f24bbc95 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:05:38.610799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f32e53fb-cfe0-48e8-b2aa-2bd5c270b08f tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95" status: 404 len: 111 microversion: 2.1 time: 0.030564 Oct 16 01:05:38.610799 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 455/930] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:38 2022] GET /compute/v2.1/servers/2f92209b-6feb-4e65-af10-6872f24bbc95 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 16 01:05:38.613073 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b97d2365-cbb2-45e0-b42d-9c6c07b4d460 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.159401 Oct 16 01:05:38.613648 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 476/931] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:38 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:38.776833 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3ace858c-7d73-46ff-b6bc-a2eee9e3119c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] Calling method '.version_select at 0x7f8bbd84cf70>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:38.800155 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3ace858c-7d73-46ff-b6bc-a2eee9e3119c tempest-TestShelveInstance-628145774 tempest-TestShelveInstance-628145774-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1829539653" status: 202 len: 0 microversion: 2.1 time: 0.027979 Oct 16 01:05:38.800744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 456/932] 10.222.0.25 () {58 vars in 1254 bytes} [Sun Oct 16 01:05:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1829539653 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:05:39.287163 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1c8952e3-9dc3-4551-851b-b6c0ccd92507 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:39.290117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1c8952e3-9dc3-4551-851b-b6c0ccd92507 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:39.374969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1c8952e3-9dc3-4551-851b-b6c0ccd92507 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:39.382404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1c8952e3-9dc3-4551-851b-b6c0ccd92507 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1093 microversion: 2.1 time: 0.104011 Oct 16 01:05:39.396533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 477/933] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:39 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1093 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:39.637525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:39.638144 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:39.645712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:39.645712 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:39.646025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:39.646727 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:39.647409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:39.647844 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:39.784246 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:39.797116 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-eea0d143-ba0a-412e-b3e9-a87471f9c68c tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.163503 Oct 16 01:05:39.798190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 457/934] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:39 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:40.411137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c9c2a58d-c8f5-463c-939b-20783aae8ac0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:40.412355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c9c2a58d-c8f5-463c-939b-20783aae8ac0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:40.495140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c9c2a58d-c8f5-463c-939b-20783aae8ac0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:40.509321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c9c2a58d-c8f5-463c-939b-20783aae8ac0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1093 microversion: 2.1 time: 0.104053 Oct 16 01:05:40.509883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 478/935] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:40 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1093 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:40.823095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:40.823703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:40.835776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:40.838492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:40.839225 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:40.842813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:40.843562 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:40.844192 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:41.059884 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:41.073622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:41.088629 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3bb937a4-06af-4c91-81bf-7fc751d609d2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1679 microversion: 2.1 time: 0.270400 Oct 16 01:05:41.098914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 458/936] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:40 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1679 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:41.535906 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5ce8a51c-1724-460b-bd20-46535aaf6433 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:41.536550 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5ce8a51c-1724-460b-bd20-46535aaf6433 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:41.672479 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5ce8a51c-1724-460b-bd20-46535aaf6433 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:41.687691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5ce8a51c-1724-460b-bd20-46535aaf6433 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1093 microversion: 2.1 time: 0.153463 Oct 16 01:05:41.689656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 479/937] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:41 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1093 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:42.115011 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:42.115542 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:42.121913 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.122477 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.122936 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.123638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.124120 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.124503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.376118 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:42.386708 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6fc7182b-351e-4628-8954-f48acee0f594 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1656 microversion: 2.1 time: 0.275434 Oct 16 01:05:42.387354 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 459/938] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:42 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1656 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:42.418409 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:42.430558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.432809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.433278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.470539 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Going to try to start instance {{(pid=119012) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 16 01:05:42.558601 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-461cf1a1-82f2-4365-a8c4-65ed6009082e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action" status: 202 len: 0 microversion: 2.1 time: 0.146197 Oct 16 01:05:42.560089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 480/939] 10.222.0.25 () {60 vars in 1278 bytes} [Sun Oct 16 01:05:42 2022] POST /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/action => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:05:42.567932 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:42.568554 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:42.602063 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.604624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.606891 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.610986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.611754 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.613502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.716805 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:42.718706 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:42.728079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.729012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.729532 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.731505 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:42.732252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:42.732926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:42.884379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:42.906329 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c4c73be9-c882-457e-8591-1d6eee5a5421 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1143 microversion: 2.1 time: 0.189826 Oct 16 01:05:42.907318 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 481/940] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:42 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1143 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:43.003230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:43.020662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ce90df7a-aeb4-4915-b9eb-5fb6045547a5 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1665 microversion: 2.1 time: 0.456504 Oct 16 01:05:43.021693 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 460/941] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:42 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1665 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:43.930362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:43.931134 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:43.938317 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:43.940045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:43.941331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:43.942258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:43.942727 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:43.944332 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:43.968171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:44.036057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:44.047026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:44.048449 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:44.049606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-21c60239-1a08-4c86-bda8-e9855ba9f27a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1209 microversion: 2.1 time: 0.122682 Oct 16 01:05:44.062903 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 482/942] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:43 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1209 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:44.099834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:44.101549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:44.102211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:44.104242 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:44.104802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:44.105312 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:44.529458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:44.538996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-db335508-aa85-4abe-97be-8a0161dea090 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1665 microversion: 2.1 time: 0.497121 Oct 16 01:05:44.539931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 461/943] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:44 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1665 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:45.116851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:45.117910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:45.130555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:45.131445 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:45.132547 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:45.134606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:45.135803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:45.137241 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:45.253679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:45.263436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d497f710-5be7-4442-8d1a-2a366e8fc5dd tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1197 microversion: 2.1 time: 0.155439 Oct 16 01:05:45.264167 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 483/944] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:45 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1197 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:45.576596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:45.578325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:45.585877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:45.586531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:45.587533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:45.588600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:45.589382 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:45.589798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:45.971798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:45.983380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-574347aa-736c-4626-a4e7-be005827d14a tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1669 microversion: 2.1 time: 0.413074 Oct 16 01:05:45.991352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 462/945] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:45 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1669 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:46.287049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:46.288185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:46.299351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:46.299849 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:46.300295 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:46.301022 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:46.301500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:46.301922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:46.812586 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:46.826928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c29c538a-2594-463f-b28a-ef8c000cd900 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.542665 Oct 16 01:05:46.833177 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 484/946] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:46 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:47.372438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7479d976-4c68-4862-8245-79dfaccf0f81", "tag": "8650d000-9eba-4333-85df-35ab0580d9c1"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:47.407405 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:47.408267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:47.420512 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:47.452802 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] Creating event network-changed:8650d000-9eba-4333-85df-35ab0580d9c1 for instance 7479d976-4c68-4862-8245-79dfaccf0f81 on hv-cinder-maste Oct 16 01:05:47.460930 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-c0540f6f-7bc7-4664-b34b-a41a5fd74094 req-b6d0dfe9-d931-4401-8212-99dae0ae0457 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096548 Oct 16 01:05:47.461669 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 463/947] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:05:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:05:47.855391 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:47.856687 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:47.863200 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:47.863959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:47.864569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:47.866363 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:47.867269 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:47.868435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:47.974669 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:48.171404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:48.184488 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-051cb99e-dca1-4d42-9d05-2900263bd5d8 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.334635 Oct 16 01:05:48.185818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 485/948] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:47 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:49.210806 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:49.217163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:49.217769 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:49.246110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:49.246619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:49.247090 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:49.247820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:49.248272 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:49.248878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:49.864765 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:49.895472 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a36fc61b-00c0-4697-9393-59135191fd29 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.685864 Oct 16 01:05:49.897158 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 464/949] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:49 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:05:50.921527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:50.922294 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:50.929920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:50.930405 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:50.930840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:50.931567 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:50.932019 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:50.932472 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:51.140931 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:51.171798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-868794e3-ebcd-4592-a863-886ac606a344 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.253944 Oct 16 01:05:51.174838 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 486/950] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:50 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:52.199412 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:52.199993 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:52.206433 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:52.206962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:52.207407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:52.208114 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:52.208573 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:52.209678 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:52.342602 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:52.350537 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a51444a-0944-4fef-bbcc-0668adec4673 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.155391 Oct 16 01:05:52.351140 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 465/951] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:52 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:53.372723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:53.373334 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:53.375198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=12 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:53.382012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:53.382487 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:53.382937 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:53.383632 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:53.384081 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:53.384498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:53.526847 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:53.535322 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b44a184f-d55b-4ddc-927a-94fe95554e1e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.166399 Oct 16 01:05:53.535928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 487/952] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:53 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:54.558296 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:54.558943 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:54.561296 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=58,nova_api:DELETE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:54.566805 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:54.567370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:54.567817 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:54.569151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:54.569834 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:54.570510 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:54.725912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:54.733583 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-99d15369-c2b2-4146-9ac1-2677051d1d06 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.179206 Oct 16 01:05:54.734180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 466/953] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:54 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:55.757324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:55.757943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:55.760338 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:05:55.766273 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:55.766773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:55.767244 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:55.767956 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:55.768415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:55.768846 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:55.917664 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:55.926258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8a3456e4-f601-46d9-8011-04f0cd2aeaa5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.172858 Oct 16 01:05:55.926855 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 488/954] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:55 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:56.950267 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:56.951078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:56.957956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:56.958540 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:56.959719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:56.960658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:56.961213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:56.961714 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:57.125649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:57.135180 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a6609d26-1dcd-4ff5-8c2c-68bc47f5e87e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.188762 Oct 16 01:05:57.151365 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 467/955] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:56 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:58.160874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:58.161588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:58.170852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:58.171557 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:58.172126 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:58.173205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:58.174085 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:58.174795 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:58.345806 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:58.355862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-93794503-7bf1-4638-8e4a-78fcb88cdb61 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.198911 Oct 16 01:05:58.356780 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 489/956] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:58 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:05:58.762744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7479d976-4c68-4862-8245-79dfaccf0f81", "name": "network-vif-plugged", "status": "completed", "tag": "8650d000-9eba-4333-85df-35ab0580d9c1"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:05:58.782453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:58.783371 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:58.784177 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:58.808843 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] Creating event network-vif-plugged:8650d000-9eba-4333-85df-35ab0580d9c1 for instance 7479d976-4c68-4862-8245-79dfaccf0f81 on hv-cinder-maste Oct 16 01:05:58.817738 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-009a1c4d-1e3d-4452-b724-eaa1f8e69e58 req-c5bb73a9-59d6-4002-88f8-1ac85fe2594f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062905 Oct 16 01:05:58.820153 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 468/957] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:05:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:05:59.379690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:05:59.380306 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:05:59.387057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:59.387561 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:59.388004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:59.388711 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:05:59.389175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:05:59.389606 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:05:59.537803 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:05:59.546063 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-e70dfc8e-646c-4726-8fbd-89ebb23557f0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.170346 Oct 16 01:05:59.546651 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 490/958] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:05:59 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:00.571883 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:00.572563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:00.590117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:00.590694 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:00.591126 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:00.591827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:00.592612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:00.593046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:00.755951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:00.764047 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e3cb54d9-6266-4d00-8b20-dcaee583b0d9 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1265 microversion: 2.1 time: 0.198019 Oct 16 01:06:00.764309 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 469/959] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:00 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1265 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:01.788276 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:01.788875 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:01.795463 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:01.795948 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:01.796377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:01.797076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:01.797525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:01.797942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:01.963890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:01.976480 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-26bc1ca8-9b82-4ebd-911b-1ce9cf0ad91a tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1451 microversion: 2.1 time: 0.192108 Oct 16 01:06:01.977460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 491/960] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:01 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1451 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:02.003062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:02.003808 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:02.012117 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:02.012703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:02.013299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:02.014210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:02.014753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:02.015384 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:02.222546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:02.232080 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6876b858-8b8a-428e-80d1-2e647ac25a20 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1451 microversion: 2.1 time: 0.232958 Oct 16 01:06:02.232685 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 470/961] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:01 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1451 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:04.231526 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7479d976-4c68-4862-8245-79dfaccf0f81", "tag": "8650d000-9eba-4333-85df-35ab0580d9c1"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:04.245858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:04.245858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:04.245858 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:04.259304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] Creating event network-changed:8650d000-9eba-4333-85df-35ab0580d9c1 for instance 7479d976-4c68-4862-8245-79dfaccf0f81 on hv-cinder-maste Oct 16 01:06:04.264691 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-1675f485-e20d-4c96-bedc-cba2f2741e90 req-b94da4b6-d14b-43af-a3b6-f3065049dac1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036829 Oct 16 01:06:04.269156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 492/962] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:12.750753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-869434324", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "networks": [{"uuid": "00f07626-fa08-4fe1-b8db-0b77245228f3"}]}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:12.759866 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 16 01:06:12.760036 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Sun Oct 16 01:06:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Oct 16 01:06:12.761807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=9 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:12.762476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=64 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:12.814151 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] validate_networks() for [('00f07626-fa08-4fe1-b8db-0b77245228f3', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:06:13.158457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:06:13.158903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:06:13.159422 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:06:13.159836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:06:13.160252 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:06:13.169208 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:13.169704 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:13.170170 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:13.170939 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:13.171401 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:13.171829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:13.192960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Getting quotas for project c322a47bb2364c8ea9250a7ecea537bf. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:06:13.197213 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Getting quotas for user 602ff9dfc79d43839abbb179cbf6f9fd and project c322a47bb2364c8ea9250a7ecea537bf. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:06:13.208835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:06:13.211203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:06:13.257753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a58b0718-3e62-4ec2-a5ee-6fd07d17c91f tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.510696 Oct 16 01:06:13.258408 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 471/963] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:06:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 512 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:06:13.271763 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-faf5abd6-e4dd-4996-b9e3-450c7ce607c6 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:13.272569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-faf5abd6-e4dd-4996-b9e3-450c7ce607c6 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:13.328877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-faf5abd6-e4dd-4996-b9e3-450c7ce607c6 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:13.336871 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-faf5abd6-e4dd-4996-b9e3-450c7ce607c6 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1218 microversion: 2.1 time: 0.070917 Oct 16 01:06:13.337448 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 493/964] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:13 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1218 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:14.360428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:14.361040 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:14.370522 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:14.371263 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:14.371705 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:14.372410 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:14.372864 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:14.373291 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:14.468458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:14.479584 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2f72971b-b05c-47d5-a32f-cae5a87d8adb tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1278 microversion: 2.1 time: 0.123471 Oct 16 01:06:14.480440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 472/965] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:14 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1278 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:15.503003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:15.503811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:15.506585 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=60 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:15.513566 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:15.514180 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:15.514629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:15.515382 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:15.515848 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:15.516281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:15.651083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:15.659115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0fff12bf-8c35-40b1-a0e5-d5b5cf089420 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.159892 Oct 16 01:06:15.659716 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 494/966] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:15 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:16.006500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1be419e-f50d-4116-aa44-9055c7014141", "tag": "030acd90-8e8f-444f-b83a-e384f9ea5bce"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:16.019395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:16.019973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:16.020601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:16.033534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] Creating event network-changed:030acd90-8e8f-444f-b83a-e384f9ea5bce for instance d1be419e-f50d-4116-aa44-9055c7014141 on hv-cinder-maste Oct 16 01:06:16.039459 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-522d8563-b663-4a4c-a528-7e5c1a0b58d0 req-11b4bb0e-1f72-406e-b0a7-427a2f7c36f9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037012 Oct 16 01:06:16.040075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 473/967] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:16.683917 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:16.684449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:16.690852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:16.691355 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:16.691788 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:16.692475 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:16.692919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:16.693485 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:16.826814 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:16.835176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d5c5b9dd-ed3c-40e3-88ef-f897e51c6591 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.155260 Oct 16 01:06:16.835756 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 495/968] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:16 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:17.858092 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:17.859438 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:17.866257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:17.866746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:17.867208 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:17.868653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:17.868653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:17.868878 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:18.004458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:18.012400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5e8a626b-0ef8-4449-bdac-2d2c4af27827 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.158563 Oct 16 01:06:18.012973 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 474/969] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:17 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:19.035800 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:19.036394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:19.043851 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:19.044334 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:19.044774 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:19.045466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:19.045914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:19.046333 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:19.176474 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:19.185361 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fbe03ce6-d884-4b1a-91f2-405ec50431b8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.153441 Oct 16 01:06:19.185964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 496/970] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:19 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:20.208981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:20.209624 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:20.216381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:20.216863 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:20.217294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:20.217992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:20.218444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:20.218903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:20.352018 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:20.360261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-826dfce2-32cb-48aa-9663-f547c7cf55da tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.155602 Oct 16 01:06:20.360831 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 475/971] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:20 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:20.843044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1be419e-f50d-4116-aa44-9055c7014141", "name": "network-vif-plugged", "status": "completed", "tag": "030acd90-8e8f-444f-b83a-e384f9ea5bce"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:20.854027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:20.854506 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:20.854963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:20.866920 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] Creating event network-vif-plugged:030acd90-8e8f-444f-b83a-e384f9ea5bce for instance d1be419e-f50d-4116-aa44-9055c7014141 on hv-cinder-maste Oct 16 01:06:20.871890 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-eff3bd63-1bf2-4a05-903a-00acd320f9af req-a93882a1-29d4-43f9-a39a-0f57f1fcf284 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032675 Oct 16 01:06:20.872775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 497/972] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:21.310812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:21.313894 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:21.321664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.322150 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.322594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.323350 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.323812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.324237 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.356228 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3ed5af4c-795e-47e6-bb6e-559e1d0ff77e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049311 Oct 16 01:06:21.356994 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 476/973] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:06:21 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:06:21.385593 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:21.386368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:21.392773 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.393369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.394059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.394873 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.395440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.396169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.426334 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:21.426829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:21.434047 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.434534 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.434992 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.435696 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.436148 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.436569 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.463528 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:06:21.543049 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:21.544330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f36ac320-bfd2-4143-96ed-688d61a97604 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 204 len: 0 microversion: 2.1 time: 0.121561 Oct 16 01:06:21.544916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 477/974] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:06:21 2022] DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:06:21.551907 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-6c4cd50a-a3d3-4680-855b-14443b50b7bd tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.171448 Oct 16 01:06:21.553293 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 498/975] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:21 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:21.554385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:21.555132 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:21.561072 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.561556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.562000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.562698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:21.563192 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:21.563622 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:21.692848 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:21.701026 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e35fa1a8-838b-45ab-b81d-604feaf77c27 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1675 microversion: 2.1 time: 0.151595 Oct 16 01:06:21.701640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 478/976] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:21 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1675 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:22.575553 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:22.576205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:22.582898 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:22.583429 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:22.584003 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:22.584924 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:22.585399 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:22.585836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:22.723699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:22.725099 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:22.725718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:22.732560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:22.732811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-86fdff09-8262-4ca5-8c4b-7101c28737b4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1318 microversion: 2.1 time: 0.160847 Oct 16 01:06:22.733059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:22.733360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 499/977] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:22 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:22.733541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:22.734211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:22.734667 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:22.735136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:22.875203 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:22.883393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d2ce9014-320b-4fa8-9565-1f8d91bc86c6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1675 microversion: 2.1 time: 0.162129 Oct 16 01:06:22.883981 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 479/978] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:22 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1675 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:23.759999 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:23.762103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:23.765546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:23.777938 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:23.778834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:23.779656 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:23.780942 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:23.781798 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:23.782605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:23.905657 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:23.906367 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:23.908211 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:23.909046 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:23.915450 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:23.915949 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:23.916393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:23.917102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:23.917565 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:23.917998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:23.920366 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:23.928904 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ab9cec7f-c8bb-4ae0-93f4-9223bfa9a597 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1508 microversion: 2.1 time: 0.176553 Oct 16 01:06:23.929489 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 500/979] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:23 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1508 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:24.056113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:24.064353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ef6c094e-3198-4216-aeda-43c923df08ca tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1675 microversion: 2.1 time: 0.162484 Oct 16 01:06:24.064762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 480/980] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:23 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1675 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:25.090221 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:25.090970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:25.098012 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:25.098587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:25.099139 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:25.099918 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:25.100441 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:25.100868 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:25.172850 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:25.189079 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ab93e8a5-f6fd-4423-80d8-2419790aa718 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 200 len: 1608 microversion: 2.1 time: 0.102896 Oct 16 01:06:25.189889 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 501/981] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:25 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 1608 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:25.488616 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f3182305-f754-4b0f-989d-7362967cb10a", "name": "network-vif-deleted", "tag": "01028fe7-7644-4a03-bb34-6762d2043774"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:25.502265 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:25.503107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:25.503538 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:25.518121 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] Creating event network-vif-deleted:01028fe7-7644-4a03-bb34-6762d2043774 for instance f3182305-f754-4b0f-989d-7362967cb10a on hv-cinder-maste Oct 16 01:06:25.523494 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-2d71c577-ea4e-45b4-92bc-86708892759f req-c2f869af-db3c-4b87-9e2c-c0e0974d3a13 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038812 Oct 16 01:06:25.524051 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 481/982] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:26.215144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:26.215775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:26.217907 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=30 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:26.223517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:26.224822 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:26.225240 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:26.226059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:26.226523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:26.227038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:26.244235 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a HTTP/1.1 Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:06:26.245936 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:06:26.247884 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'f3182305-f754-4b0f-989d-7362967cb10a'} Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:06:26.248955 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:06:26.250525 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:06:26.252080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:06:26.252080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1a3ab152-62d2-4cd1-9a1c-49d91b419431 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 404 len: 111 microversion: 2.1 time: 0.036406 Oct 16 01:06:26.252080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 502/983] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:26 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:06:26.257245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-82d23233-4596-4a72-ae64-3502cf14d53b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '.version_select at 0x7f8bbd8fad30>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:26.278129 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-82d23233-4596-4a72-ae64-3502cf14d53b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1914944121" status: 202 len: 0 microversion: 2.1 time: 0.025182 Oct 16 01:06:26.278703 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 482/984] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:06:26 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1914944121 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:06:26.987681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-35586b55-f35c-436f-9ee1-8ee3caf05804 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: .version_select at 0x7f8bbdb7d160>, body: {"keypair": {"name": "tempest-keypair-2091634348", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHuwuiDAa3SNpVfyMjNdZl9HFezFyBDoaZuX5vuZliVDf4iQAS0/+8TAi0YzR2F1vWXy2ofh/ktpYq3voUEwBP4jOz+kSQ7uUxw/BbXzaz5Fah7zpYgTilRgmr2BxPch5w=="}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:27.003609 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-35586b55-f35c-436f-9ee1-8ee3caf05804 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for project b8852bfe31aa4ba68190661d26c177cb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:06:27.008145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-35586b55-f35c-436f-9ee1-8ee3caf05804 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for user e1283100ac2c439cb308b9c57ae31f40 and project b8852bfe31aa4ba68190661d26c177cb. Resources: {'key_pairs'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:06:27.031568 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-35586b55-f35c-436f-9ee1-8ee3caf05804 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.047740 Oct 16 01:06:27.032165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 503/985] 10.222.0.25 () {60 vars in 1159 bytes} [Sun Oct 16 01:06:26 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:28.358103 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-827716180", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "57335160-65ed-4241-a862-dbf37da52fd2"}], "security_groups": [{"name": "tempest-securitygroup--905271810"}], "key_name": "tempest-keypair-2091634348", "user_data": "}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:28.465432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.network.neutron [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] validate_networks() for [('57335160-65ed-4241-a862-dbf37da52fd2', None, None, None, None, None)] {{(pid=119013) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:06:28.849718 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Flavor limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:06:28.850137 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Image limits 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:06:28.850646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Flavor pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:06:28.851091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Image pref 0:0:0 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:06:28.851570 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.virt.hardware [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119013) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:06:28.860631 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:28.861125 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:28.861566 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:28.862303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:28.862762 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:28.863216 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:28.880986 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for project b8852bfe31aa4ba68190661d26c177cb. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:06:28.885183 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.quota [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Getting quotas for user e1283100ac2c439cb308b9c57ae31f40 and project b8852bfe31aa4ba68190661d26c177cb. Resources: {'cores', 'instances', 'ram'} {{(pid=119013) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:06:28.897796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Going to run 1 instances... {{(pid=119013) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:06:28.900322 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119013) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:06:28.950594 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-caace4ad-4787-4cf8-9a74-96569ec6f335 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.596712 Oct 16 01:06:28.951502 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 483/986] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:06:28 2022] POST /compute/v2.1/servers => generated 401 bytes in 598 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:06:28.963093 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0893b777-3a5a-494c-85a5-ae9786cf0414 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:28.963729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0893b777-3a5a-494c-85a5-ae9786cf0414 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:29.023240 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0893b777-3a5a-494c-85a5-ae9786cf0414 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:29.030443 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0893b777-3a5a-494c-85a5-ae9786cf0414 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1237 microversion: 2.1 time: 0.071986 Oct 16 01:06:29.031038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 504/987] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:28 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1237 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:06:30.053786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:30.054445 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:30.061885 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:30.062413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:30.062873 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:30.063598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:30.064095 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:30.064525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:30.138914 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:30.151906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6f0da26c-4d55-4c2d-8f5f-071cc1bbce1b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1289 microversion: 2.1 time: 0.102589 Oct 16 01:06:30.152503 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 484/988] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:30 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1289 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:31.178004 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:31.179922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:31.186699 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:31.187770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:31.187770 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:31.188562 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:31.189089 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:31.189758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:31.333992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:31.345877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-95194fe2-0628-47e6-b9b9-e3047cd8d8a3 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.172379 Oct 16 01:06:31.346751 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 505/989] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:31 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:31.839505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "78ed622b-285f-4f9b-a373-bdfc9b667fc6", "tag": "b75f86c7-a0ac-43e2-8da9-10f736efe195"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:31.851881 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:31.852370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:31.853642 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:31.868567 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] Creating event network-changed:b75f86c7-a0ac-43e2-8da9-10f736efe195 for instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 on hv-cinder-maste Oct 16 01:06:31.874773 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-467b7011-9375-4c7b-9903-b9cdf12cbce4 req-e2c2de00-1469-4ecc-ac2e-441e4cfedd13 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041099 Oct 16 01:06:31.875724 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 485/990] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:32.370575 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:32.371422 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:32.377945 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:32.378433 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:32.378901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:32.379600 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:32.380062 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:32.380490 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:32.515157 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:32.523527 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-54b328a5-7748-41f1-9d48-c27f610110fa tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.156710 Oct 16 01:06:32.524116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 506/991] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:32 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:33.547627 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:33.548548 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:33.554786 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:33.555318 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:33.555758 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:33.556461 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:33.556925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:33.557356 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:33.764900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:33.773210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-10ecc909-e76e-4e2d-84d7-dc12f3d6acac tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.230271 Oct 16 01:06:33.773779 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 486/992] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:33 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:34.796238 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:34.796845 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:34.804074 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:34.804558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:34.804991 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:34.805768 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:34.806149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:34.806570 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:34.949047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:34.963313 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-81ec5c7c-bcd5-4476-8c2a-e8dcc2e486dc tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.170598 Oct 16 01:06:34.967249 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 507/993] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:34 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:35.987381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:35.987991 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:35.995134 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:35.995753 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:35.996035 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:35.996744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:35.997196 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:35.997633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:36.132015 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:36.139795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-d3667529-e5c6-4044-acfc-02ec0f176b7f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.156470 Oct 16 01:06:36.140400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 487/994] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:35 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:36.933218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "78ed622b-285f-4f9b-a373-bdfc9b667fc6", "name": "network-vif-plugged", "status": "completed", "tag": "b75f86c7-a0ac-43e2-8da9-10f736efe195"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:36.944077 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:36.944645 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:36.945330 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:36.956745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] Creating event network-vif-plugged:b75f86c7-a0ac-43e2-8da9-10f736efe195 for instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 on hv-cinder-maste Oct 16 01:06:36.962281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-4385078e-4106-4108-b188-1a5e39c1f120 req-587d7c8b-6c40-4e5b-970a-ab8a0cfc6ab3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032943 Oct 16 01:06:36.962893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 508/995] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:37.163078 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:37.164933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:37.170454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:37.171059 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:37.171635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:37.172353 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:37.172812 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:37.173243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:37.318556 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:37.327073 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-7a921e8b-17d9-4e65-b839-2df19b8e9fb9 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.168773 Oct 16 01:06:37.327641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 488/996] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:37 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:38.351435 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:38.352100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:38.358242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:38.358729 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:38.359197 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:38.359891 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:38.360349 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:38.360775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:38.490302 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:38.499106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-efb0ab8c-3ac6-4173-8e5d-6263cd164dd6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1362 microversion: 2.1 time: 0.152476 Oct 16 01:06:38.499700 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 509/997] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:38 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:39.210243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4e1d6e34-f5bc-4ea3-ba14-6cd5d2d36659", "device": "/dev/sdb"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:39.213181 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:39.215677 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:39.216346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=6 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:39.223253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.223746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.224186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.225641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.226247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.226716 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.355201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:39.470187 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-635fd022-d8f9-4f04-b701-2895520e0958 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "POST /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.264931 Oct 16 01:06:39.470828 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 489/998] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:06:39 2022] POST /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141/os-volume_attachments => generated 194 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:39.531428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:39.532264 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:39.534297 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:06:39.540535 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.541021 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.541449 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.542140 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.542590 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.543045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.681882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:06:39.689647 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0fa42183-a1a5-45e2-b619-0a640ea9c468 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1548 microversion: 2.1 time: 0.170553 Oct 16 01:06:39.690223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 510/999] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:06:39 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1548 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:06:39.714337 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:06:39.714997 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:06:39.721191 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.721672 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.722107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.722794 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:39.723277 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:39.723711 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:39.814632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6a8b97ed-86ef-498e-9192-1f3d0190bbba tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.105553 Oct 16 01:06:39.815385 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 490/1000] 10.222.0.25 () {58 vars in 1275 bytes} [Sun Oct 16 01:06:39 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:06:40.752045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "78ed622b-285f-4f9b-a373-bdfc9b667fc6", "tag": "b75f86c7-a0ac-43e2-8da9-10f736efe195"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:06:40.762516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:06:40.763025 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:06:40.763464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:06:40.775242 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] Creating event network-changed:b75f86c7-a0ac-43e2-8da9-10f736efe195 for instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 on hv-cinder-maste Oct 16 01:06:40.780740 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-11223916-2c1a-4b6b-94c3-13825159a121 req-6ed19074-9a05-41d0-afff-64d9cef32963 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032306 Oct 16 01:06:40.781362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 511/1001] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:06:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:03.213315 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=6,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:03.214066 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=8,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:03.218280 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:03.218807 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:03.226662 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:03.227162 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:03.227593 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:03.228330 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:03.228776 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:03.229200 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:03.313492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lazy-loading 'services' on Instance uuid d1be419e-f50d-4116-aa44-9055c7014141 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:07:03.385274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-3e85623f-d844-4dc6-91af-f25988ca30d9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141/os-volume_attachments/4e1d6e34-f5bc-4ea3-ba14-6cd5d2d36659" status: 202 len: 0 microversion: 2.1 time: 0.173810 Oct 16 01:07:03.385857 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 491/1002] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:07:03 2022] DELETE /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141/os-volume_attachments/4e1d6e34-f5bc-4ea3-ba14-6cd5d2d36659 => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:07:04.513446 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:04.514027 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:04.516206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:04.516854 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=80 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:04.524104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:04.524615 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:04.525080 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:04.525775 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:04.526229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:04.526655 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:04.551116 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:07:04.627165 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f2a226ff-4826-461b-b628-c58c6e2b7bd9 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 204 len: 0 microversion: 2.1 time: 0.117717 Oct 16 01:07:04.627757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 512/1003] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:07:04 2022] DELETE /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:07:04.638111 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:04.638818 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:04.644948 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:04.645482 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:04.645931 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:04.646626 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:04.647115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:04.647539 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:04.781525 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:04.790009 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-950f17ee-bb89-4bd2-91a4-15a25bfda779 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1514 microversion: 2.1 time: 0.156737 Oct 16 01:07:04.790588 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 492/1004] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:04 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:05.814496 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:05.815218 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:05.821899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:05.822377 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:05.822808 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:05.823517 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:05.823966 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:05.824388 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:05.961715 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:05.969686 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-db915b78-4e2c-40dc-9143-af54fdc7b4b5 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1514 microversion: 2.1 time: 0.159389 Oct 16 01:07:05.970320 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 513/1005] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:05 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1514 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:06.964820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1be419e-f50d-4116-aa44-9055c7014141", "name": "network-vif-deleted", "tag": "030acd90-8e8f-444f-b83a-e384f9ea5bce"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:06.978377 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:06.978757 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:06.979224 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:06.991925 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] Creating event network-vif-deleted:030acd90-8e8f-444f-b83a-e384f9ea5bce for instance d1be419e-f50d-4116-aa44-9055c7014141 on hv-cinder-maste Oct 16 01:07:06.994050 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:06.994713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:06.997751 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-fcc47c32-52c8-4d3f-817d-929580b7b4b8 req-b303905f-bc45-4efc-bd26-445c9587a5d7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036712 Oct 16 01:07:06.998551 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 493/1006] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:07.001951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:07.002434 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:07.002971 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:07.003589 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:07.004047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:07.004470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:07.073661 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:07.081590 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-1fbbb879-9e29-4a7a-b038-87f008346270 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 200 len: 1303 microversion: 2.1 time: 0.091923 Oct 16 01:07:07.082175 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 514/1007] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:06 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 1303 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:08.106186 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:08.106902 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: d1be419e-f50d-4116-aa44-9055c7014141] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:08.113453 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:08.114487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:08.114956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:08.115683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:08.116139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:08.116563 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:08.130444 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] HTTP exception thrown: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Request method failure captured: Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 HTTP/1.1 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1kD04V35otRakb9Ppzy-1tN67lkqTeUMwDUho28nP5g8wHex5wwygbyKP9Dl_i2_iFdSz7atLLxGTDK_kfHZOANFoHflkZsIKO9YklQxfQQX5CUvwkNfrTWLCPAftJxd4MIf43oRTvKBSHCe2shmIf5LX6eDE3bOVE_Af5pN2gIDm0 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: c322a47bb2364c8ea9250a7ecea537bf Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:07:08.132188 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c322a47bb2364c8ea9250a7ecea537bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c322a47bb2364c8ea9250a7ecea537bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c322a47bb2364c8ea9250a7ecea537bf", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c322a47bb2364c8ea9250a7ecea537bf"}]}] Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: c322a47bb2364c8ea9250a7ecea537bf Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-VolumesSnapshotTestJSON-799418870-project Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: 602ff9dfc79d43839abbb179cbf6f9fd Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-VolumesSnapshotTestJSON-799418870-project Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. Oct 16 01:07:08.134037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'd1be419e-f50d-4116-aa44-9055c7014141'} Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:08.135110 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. Oct 16 01:07:08.136849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:08.138319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Returning 404 to user: Instance d1be419e-f50d-4116-aa44-9055c7014141 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:08.138319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f96bc6f6-4155-4fb0-aa99-9d10192f221b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141" status: 404 len: 111 microversion: 2.1 time: 0.031669 Oct 16 01:07:08.138319 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 494/1008] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:08 2022] GET /compute/v2.1/servers/d1be419e-f50d-4116-aa44-9055c7014141 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:17.975341 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-866670939", "imageRef": "76b907c1-15da-438d-b27a-e8e47a9813dc", "flavorRef": "452", "networks": [{"uuid": "00f07626-fa08-4fe1-b8db-0b77245228f3"}]}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:17.984914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:17.985757 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:18.198630 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.network.neutron [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] validate_networks() for [('00f07626-fa08-4fe1-b8db-0b77245228f3', None, None, None, None, None)] {{(pid=119012) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 16 01:07:18.547874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Flavor limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 16 01:07:18.548315 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Image limits 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 16 01:07:18.548796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Flavor pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 16 01:07:18.549210 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Image pref 0:0:0 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 16 01:07:18.549626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.virt.hardware [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=119012) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 16 01:07:18.558982 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:18.559470 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:18.559914 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:18.560649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:18.561109 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:18.561540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:18.579266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Getting quotas for project c322a47bb2364c8ea9250a7ecea537bf. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 16 01:07:18.583261 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.quota [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Getting quotas for user 602ff9dfc79d43839abbb179cbf6f9fd and project c322a47bb2364c8ea9250a7ecea537bf. Resources: {'cores', 'instances', 'ram'} {{(pid=119012) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 16 01:07:18.594758 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Going to run 1 instances... {{(pid=119012) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 16 01:07:18.598138 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='76b907c1-15da-438d-b27a-e8e47a9813dc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=119012) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 16 01:07:18.646373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8e963c39-3407-4ee9-8829-746bdb3aff82 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.674883 Oct 16 01:07:18.647230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 515/1009] 10.222.0.25 () {60 vars in 1147 bytes} [Sun Oct 16 01:07:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 676 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 16 01:07:18.658325 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a6d2b9ef-f1d4-4e58-bc5c-e0cde8246962 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:18.658988 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a6d2b9ef-f1d4-4e58-bc5c-e0cde8246962 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:18.661201 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=8 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:18.661889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:18.779600 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a6d2b9ef-f1d4-4e58-bc5c-e0cde8246962 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:18.787038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a6d2b9ef-f1d4-4e58-bc5c-e0cde8246962 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1218 microversion: 2.1 time: 0.132831 Oct 16 01:07:18.787601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 495/1010] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:18 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1218 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:07:19.812286 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:19.813065 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:19.822101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:19.822587 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:19.823206 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:19.823979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:19.824494 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:19.824986 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:19.894156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:19.903654 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4d1f9151-a479-471f-88c4-83f29a161e7a tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1288 microversion: 2.1 time: 0.095933 Oct 16 01:07:19.904380 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 516/1011] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:19 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1288 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:20.927670 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:20.928314 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:20.936417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:20.936912 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:20.937343 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:20.938041 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:20.938488 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:20.938956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:21.163016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:21.171803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2113c868-f1ee-4ac6-9bbb-1570f18b566c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.247899 Oct 16 01:07:21.172770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 496/1012] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:20 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:21.419816 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38ce2d6c-6685-47ba-954e-94c647f15f86", "tag": "31064445-732b-4b77-9363-f66edf178073"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:21.432529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:21.433195 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:21.433693 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:21.445639 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] Creating event network-changed:31064445-732b-4b77-9363-f66edf178073 for instance 38ce2d6c-6685-47ba-954e-94c647f15f86 on hv-cinder-maste Oct 16 01:07:21.450736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-787a0e3f-ff44-4c20-9b44-0ded3fa08379 req-d1d5c570-cb9d-4cc1-bcf4-1d7727a0076a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034930 Oct 16 01:07:21.451912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 517/1013] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:22.197641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:22.198240 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:22.204847 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:22.205336 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:22.205772 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:22.206473 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:22.206962 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:22.207394 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:22.341222 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:22.348840 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-db752ae2-04d6-4e69-bf23-c311e2d0c85b tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.156063 Oct 16 01:07:22.349402 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 497/1014] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:22 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:23.373088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:23.373713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:23.380331 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:23.380821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:23.381263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:23.381969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:23.382427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:23.382879 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:23.522351 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:23.531037 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f1410bed-81f4-422a-b633-b3699dc53267 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.163042 Oct 16 01:07:23.531707 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 518/1015] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:23 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:24.554047 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:24.554697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:24.562531 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:24.563056 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:24.563496 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:24.564195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:24.564653 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:24.565075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:24.701115 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:24.708921 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a3a4b00a-cf31-4aac-9155-7e670aeb39fc tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.158782 Oct 16 01:07:24.709487 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 498/1016] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:24 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:25.734146 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:25.734794 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:25.741311 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:25.741797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:25.742230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:25.743563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:25.744097 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:25.744516 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:25.874440 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:25.882187 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-fe023cc2-3b4b-483d-83b4-d4db9a232705 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.152462 Oct 16 01:07:25.882766 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 519/1017] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:25 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:26.905905 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:26.906511 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:26.912908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:26.913393 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:26.913828 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:26.914518 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:26.915086 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:26.915513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:27.044091 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:27.051803 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-add35a1b-f38b-4318-a764-568265e7a3d3 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.149811 Oct 16 01:07:27.052362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 499/1018] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:26 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:27.635784 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38ce2d6c-6685-47ba-954e-94c647f15f86", "name": "network-vif-plugged", "status": "completed", "tag": "31064445-732b-4b77-9363-f66edf178073"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:27.646427 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:27.647028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:27.647452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:27.659230 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] Creating event network-vif-plugged:31064445-732b-4b77-9363-f66edf178073 for instance 38ce2d6c-6685-47ba-954e-94c647f15f86 on hv-cinder-maste Oct 16 01:07:27.664176 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-c7bdd27c-0836-4fe5-929a-a15c62a25a9c req-3b4f7af8-1c76-4c5a-b00b-39c0649c2fb7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032848 Oct 16 01:07:27.664769 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 520/1019] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:28.075264 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:28.075877 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:28.082295 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:28.082800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:28.083250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:28.083945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:28.084409 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:28.084836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:28.219787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:28.227596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-54c7fed9-7772-44b6-a5af-03d429a20c9e tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1318 microversion: 2.1 time: 0.156729 Oct 16 01:07:28.228176 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 500/1020] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:28 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:29.247776 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=2 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:29.248541 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell0:SELECT=6 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:29.253536 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:29.254229 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:29.261394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:29.261818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:29.262252 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:29.262964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:29.263415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:29.263836 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:29.398887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:29.407921 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ee21d751-6361-46b7-9fee-aae30eff48af tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1507 microversion: 2.1 time: 0.161057 Oct 16 01:07:29.408510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 521/1021] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:29 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1507 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:29.432243 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "85aba317-2840-48e5-b08f-bebe133c00a5", "device": "/dev/sdb"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:29.437294 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:29.440346 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell0:SELECT=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:29.447501 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:29.448282 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:29.448719 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:29.449436 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:29.449892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:29.450527 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:29.703355 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0218267b-bf45-43ca-8188-487dfc9190d8 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "POST /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.275309 Oct 16 01:07:29.703975 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 501/1022] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:07:29 2022] POST /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86/os-volume_attachments => generated 194 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:36.029395 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:36.029974 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:36.036964 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.037713 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.038199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.039018 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.039509 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.039943 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.072360 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-0d3293fb-a707-4081-b6a9-a868e7effa94 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/ips" status: 200 len: 146 microversion: 2.1 time: 0.047121 Oct 16 01:07:36.091278 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 522/1023] 10.222.0.25 () {58 vars in 1248 bytes} [Sun Oct 16 01:07:36 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/ips => generated 146 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:36.558588 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7479d976-4c68-4862-8245-79dfaccf0f81", "tag": "8650d000-9eba-4333-85df-35ab0580d9c1"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:36.566628 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:36.567217 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:36.574969 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.575464 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.575908 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.576619 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.577079 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.577506 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.585327 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.585809 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.586237 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.598304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] Creating event network-changed:8650d000-9eba-4333-85df-35ab0580d9c1 for instance 7479d976-4c68-4862-8245-79dfaccf0f81 on hv-cinder-maste Oct 16 01:07:36.604475 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-583fdb4e-7961-4e8e-be02-268b86f41372 req-f33fdd09-7e2f-4ba3-97e6-028564fc7d97 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050064 Oct 16 01:07:36.605117 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 523/1024] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:36.616758 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:07:36.717775 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c2a31baa-71c8-43e6-88c6-bb040515880e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 204 len: 0 microversion: 2.1 time: 0.161110 Oct 16 01:07:36.718697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 502/1025] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:07:36 2022] DELETE /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:07:36.727266 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:36.728933 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:36.737038 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.737821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.738569 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.741782 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:36.742420 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:36.743060 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:36.893597 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:36.901741 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-03a926e9-5aa8-41fb-9bf7-3ad0fd985557 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1576 microversion: 2.1 time: 0.178631 Oct 16 01:07:36.902452 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 524/1026] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:36 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1576 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:37.459796 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "aaeb0149-234d-4fc6-9f49-732bc4872efd"}} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:37.462597 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:37.470027 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:37.470476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:37.470942 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:37.471648 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:37.472107 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:37.472664 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:37.927159 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:37.927701 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:37.934270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:37.934753 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:37.935216 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:37.935912 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:37.936370 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:37.936797 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:38.066627 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:38.075559 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ae18c351-b967-4709-9c76-5ab148ad4da6 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1576 microversion: 2.1 time: 0.152388 Oct 16 01:07:38.076132 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 525/1027] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:37 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1576 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:07:39.099101 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:39.100006 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:39.107963 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:39.108456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:39.108901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:39.109616 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:39.110076 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:39.110507 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:39.261629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:39.270088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-decfc974-9c10-488e-9e9d-155d346c9d54 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1576 microversion: 2.1 time: 0.175062 Oct 16 01:07:39.270690 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 526/1028] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:39 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1576 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:40.293558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:40.294260 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:40.300901 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:40.301387 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:40.301823 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:40.302518 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:40.303015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:40.303428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:40.438103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:40.446268 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-82d50cb4-7f17-4e2f-880a-af59bf742166 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1576 microversion: 2.1 time: 0.157282 Oct 16 01:07:40.449375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 527/1029] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:40 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1576 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:07:40.684010 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-571b652a-dad6-487e-8873-124c7d9ab44e tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.228477 Oct 16 01:07:40.684612 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 503/1030] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:07:37 2022] POST /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 194 bytes in 3230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:41.476217 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:41.477045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:41.484555 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:41.485047 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:41.485546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:41.486478 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:41.489604 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:41.490059 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:41.531533 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7479d976-4c68-4862-8245-79dfaccf0f81", "name": "network-vif-deleted", "tag": "8650d000-9eba-4333-85df-35ab0580d9c1"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:41.546070 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:41.547169 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:41.547623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:41.562836 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] Creating event network-vif-deleted:8650d000-9eba-4333-85df-35ab0580d9c1 for instance 7479d976-4c68-4862-8245-79dfaccf0f81 on hv-cinder-maste Oct 16 01:07:41.572910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-f828a9a9-44fe-4bfa-b0e5-d89cc74e017a req-f4d15d5d-c99a-4f17-9152-d1794b64b90e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046160 Oct 16 01:07:41.574190 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 504/1031] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:07:41.619510 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:41.630840 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-25649a72-accd-482c-9227-85e2617f077c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1389 microversion: 2.1 time: 0.158483 Oct 16 01:07:41.631629 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 528/1032] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:41 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1389 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:42.656640 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:42.657324 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:42.663945 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:42.664425 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:42.664858 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:42.666148 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:42.666601 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:42.667055 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:42.697253 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:42.697650 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:42.698081 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:42.698513 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lazy-loading 'fault' on Instance uuid 7479d976-4c68-4862-8245-79dfaccf0f81 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:07:42.741357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:42.749639 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9bb80ab1-5884-4924-b1c3-6579f5ea3b1c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1229 microversion: 2.1 time: 0.097238 Oct 16 01:07:42.750222 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 505/1033] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:42 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1229 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:07:43.772866 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:43.773375 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:43.779893 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:43.780381 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:43.780818 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:43.781523 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:43.781979 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:43.782630 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:43.816927 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:43.817413 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:43.817856 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:43.818304 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lazy-loading 'fault' on Instance uuid 7479d976-4c68-4862-8245-79dfaccf0f81 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:07:43.858621 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:43.866181 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-5fad089a-ecd1-440d-92e4-463eae0ff29e tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1229 microversion: 2.1 time: 0.097300 Oct 16 01:07:43.866751 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 529/1034] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:43 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1229 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:44.889996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:44.890625 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:44.897083 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:44.897568 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:44.898006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:44.898700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:44.899210 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:44.899977 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:44.931131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:44.931795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:44.932298 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:44.932700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lazy-loading 'fault' on Instance uuid 7479d976-4c68-4862-8245-79dfaccf0f81 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:07:44.977212 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:44.987741 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-0ac9f6c0-3fdc-4a73-ba51-62fe6b29c40c tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 200 len: 1229 microversion: 2.1 time: 0.101192 Oct 16 01:07:44.989055 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 506/1035] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:44 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 1229 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:46.012417 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:46.013467 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 7479d976-4c68-4862-8245-79dfaccf0f81] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:46.020860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:46.021369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:46.021899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:46.022530 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:46.023044 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:46.023481 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:46.039106 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] HTTP exception thrown: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Request method failure captured: Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 HTTP/1.1 Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1eG2j1-Qx4qQKlflgNnJ9o38mFemkYM9ej90vtQx4uBAfNoUgZe7Ad3_Gzmuz7XTwYgWUxXLq9IS5Op3iLbi9EUTnQj-YCg4Dc7hD7cRcPJpMaZq-g3E1x85xxWVY1RJ1q7Y9J8JNMmZVQ3NEpMJG2ruwTnOIZZdu0eVzhJOk8c_1c Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:07:46.040811 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cb1743385ea14cc39244c83e8fc88e7e", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cb1743385ea14cc39244c83e8fc88e7e"}]}] Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: c4e65cef89af48a7a51e5548634de75f Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. Oct 16 01:07:46.043247 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '7479d976-4c68-4862-8245-79dfaccf0f81'} Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:46.045042 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. Oct 16 01:07:46.047287 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:46.049778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Returning 404 to user: Instance 7479d976-4c68-4862-8245-79dfaccf0f81 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:46.049778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-4ff00e69-fa35-4748-bb9d-b280bcc31df0 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81" status: 404 len: 111 microversion: 2.1 time: 0.034534 Oct 16 01:07:46.049778 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 530/1036] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:46 2022] GET /compute/v2.1/servers/7479d976-4c68-4862-8245-79dfaccf0f81 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:48.284418 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:48.286701 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:48.293440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:48.293933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:48.294381 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:48.295113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:48.295574 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:48.296013 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:48.330795 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-fb6625d3-2573-4c56-b13e-7dc112c529b4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050251 Oct 16 01:07:48.331900 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 507/1037] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:07:48 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:48.343143 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:48.343733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:48.347466 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_api:SELECT=46,nova_api:INSERT=3 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:48.351028 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:48.351528 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:48.351968 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:48.352681 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:48.353354 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:48.353821 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:48.357039 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=77 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:48.387276 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b5318b5b-3113-4787-8929-f87ac3adf60d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd" status: 200 len: 194 microversion: 2.1 time: 0.048749 Oct 16 01:07:48.387886 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 531/1038] 10.222.0.25 () {58 vars in 1413 bytes} [Sun Oct 16 01:07:48 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:49.189756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:49.190263 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:49.192348 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:49.199284 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.199787 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.200234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.200966 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.201426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.201876 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.205687 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:07:49.292782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.objects.instance [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lazy-loading 'services' on Instance uuid 38ce2d6c-6685-47ba-954e-94c647f15f86 {{(pid=119013) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:07:49.377910 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9be47c87-b10d-4100-b7f3-f9a015ebeeab tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "DELETE /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86/os-volume_attachments/85aba317-2840-48e5-b08f-bebe133c00a5" status: 202 len: 0 microversion: 2.1 time: 0.192158 Oct 16 01:07:49.378697 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 508/1039] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:07:49 2022] DELETE /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86/os-volume_attachments/85aba317-2840-48e5-b08f-bebe133c00a5 => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:07:49.547190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:49.547683 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:49.550437 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "391edabf-85a9-4565-92f6-ebd0664d3e20", "tag": "ea701c3f-eb65-4a82-83d0-b757b0f39524"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:49.554137 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.554626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.555092 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.555796 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.556255 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.556703 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.561816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.562388 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.562920 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.574764 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] Creating event network-changed:ea701c3f-eb65-4a82-83d0-b757b0f39524 for instance 391edabf-85a9-4565-92f6-ebd0664d3e20 on hv-cinder-maste Oct 16 01:07:49.579940 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-625f0cb4-f765-426b-b861-0c17ca01aea0 req-c866f078-ad5c-46d7-88e2-4ae6f831804d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033905 Oct 16 01:07:49.580813 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 509/1040] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:49.585882 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Going to try to terminate instance {{(pid=119012) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:07:49.668359 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f5bb4505-1244-4bc6-afc8-2d0042e7290b tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "DELETE /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 204 len: 0 microversion: 2.1 time: 0.124428 Oct 16 01:07:49.669189 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 532/1041] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:07:49 2022] DELETE /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:07:49.677965 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:49.678756 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:49.685440 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.685933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.686407 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.687136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.687596 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.688025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.770978 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a43c05f3-8f7c-4226-9d8d-014c11f25285"}} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:49.773456 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:49.779993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.780483 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.780926 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.781636 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:49.782088 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:49.782512 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:49.902113 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:49.910666 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-e4bdecac-055d-4376-aa59-0af62ed7b933 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1575 microversion: 2.1 time: 0.236801 Oct 16 01:07:49.911417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 510/1042] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:49 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1575 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:50.936661 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:50.937530 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:50.944274 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:50.944739 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:50.945171 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:50.945865 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:50.946318 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:50.946736 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:51.144800 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:51.156019 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-748586ad-c77c-4436-a64e-a383d13ce0b5 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1575 microversion: 2.1 time: 0.224352 Oct 16 01:07:51.157466 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 511/1043] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:50 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1575 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 16 01:07:51.235334 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-3d894565-fe1b-45a2-b9a1-34c63df95ac7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "POST /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.468587 Oct 16 01:07:51.236223 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 533/1044] 10.222.0.25 () {60 vars in 1323 bytes} [Sun Oct 16 01:07:49 2022] POST /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 194 bytes in 1470 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 16 01:07:51.580820 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:51.581313 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:51.587236 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:51.587721 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:51.588207 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:51.588906 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:51.589362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:51.589782 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:51.616492 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:07:51.700691 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-85b207c0-eabd-49e3-95a8-5026a1a8de7c tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "DELETE /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 204 len: 0 microversion: 2.1 time: 0.123426 Oct 16 01:07:51.702075 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 512/1045] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:07:51 2022] DELETE /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 16 01:07:51.711670 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:51.713052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:51.720733 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:51.721290 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:51.721860 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:51.722685 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:51.723428 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:51.723951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:51.871970 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:51.882877 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-d265eae9-1861-4e2e-9443-fc6dcb8b86f4 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1513 microversion: 2.1 time: 0.175597 Oct 16 01:07:51.884502 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 534/1046] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:51 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1513 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:52.128633 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "391edabf-85a9-4565-92f6-ebd0664d3e20", "name": "network-vif-deleted", "tag": "ea701c3f-eb65-4a82-83d0-b757b0f39524"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:52.139400 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:52.139889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:52.140360 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:52.153269 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] Creating event network-vif-deleted:ea701c3f-eb65-4a82-83d0-b757b0f39524 for instance 391edabf-85a9-4565-92f6-ebd0664d3e20 on hv-cinder-maste Oct 16 01:07:52.158347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-9c278a28-d020-4d42-b477-adc3e9503890 req-07c272b0-57ca-4aa3-a362-8c2adfc04fd4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033987 Oct 16 01:07:52.159000 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 513/1047] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:52.181969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:52.182596 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:52.189250 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:52.189735 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:52.190379 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:52.191119 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:52.191581 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:52.192007 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:52.258726 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:52.267169 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-ae9be75c-a79a-44a4-8261-bfae6de68d74 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1507 microversion: 2.1 time: 0.089305 Oct 16 01:07:52.267750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 535/1048] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:52 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1507 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:52.910332 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:52.911062 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:52.918849 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:52.919366 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:52.919816 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:52.920523 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:52.920982 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:52.921413 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:53.062829 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:53.071038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-c1bd9583-fc75-4db8-be8f-c6ac8997b2ee tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1513 microversion: 2.1 time: 0.164987 Oct 16 01:07:53.071603 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 514/1049] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:52 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1513 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:53.292070 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:53.292724 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:53.299227 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:53.299710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:53.300156 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:53.300839 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:53.301321 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:53.301744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:53.369910 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4e65cef89af48a7a51e5548634de75f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb1743385ea14cc39244c83e8fc88e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:53.378030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a2cca4f8-bca7-46ea-b8bf-232ce43ba656 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 200 len: 1224 microversion: 2.1 time: 0.090476 Oct 16 01:07:53.378626 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 536/1050] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:53 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:54.096546 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:54.097607 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:54.106825 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.107772 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.108598 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.109634 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.110541 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.111276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.178190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38ce2d6c-6685-47ba-954e-94c647f15f86", "name": "network-vif-deleted", "tag": "31064445-732b-4b77-9363-f66edf178073"}]} {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:07:54.191919 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '602ff9dfc79d43839abbb179cbf6f9fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c322a47bb2364c8ea9250a7ecea537bf', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:07:54.195150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.196567 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.197002 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.203641 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-ab05d599-3a2c-4656-be33-9ac9db8f7e1d tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 200 len: 1303 microversion: 2.1 time: 0.112288 Oct 16 01:07:54.204742 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 515/1051] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:54 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 1303 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:07:54.223574 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.compute.server_external_events [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] Creating event network-vif-deleted:31064445-732b-4b77-9363-f66edf178073 for instance 38ce2d6c-6685-47ba-954e-94c647f15f86 on hv-cinder-maste Oct 16 01:07:54.230723 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [req-5ea058c3-4d14-4c42-a5ee-b50c53488721 req-f9fc58d5-9f9c-4918-a55f-20912f5a500d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058861 Oct 16 01:07:54.231903 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 537/1052] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:07:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:07:54.401651 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:54.402262 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: 391edabf-85a9-4565-92f6-ebd0664d3e20] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:54.408769 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.409250 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.409683 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.410379 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.410830 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.411285 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.426163 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] HTTP exception thrown: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Request method failure captured: Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 HTTP/1.1 Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1eG2j1-Qx4qQKlflgNnJ9o38mFemkYM9ej90vtQx4uBAfNoUgZe7Ad3_Gzmuz7XTwYgWUxXLq9IS5Op3iLbi9EUTnQj-YCg4Dc7hD7cRcPJpMaZq-g3E1x85xxWVY1RJ1q7Y9J8JNMmZVQ3NEpMJG2ruwTnOIZZdu0eVzhJOk8c_1c Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:07:54.427896 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cb1743385ea14cc39244c83e8fc88e7e", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cb1743385ea14cc39244c83e8fc88e7e"}]}] Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: c4e65cef89af48a7a51e5548634de75f Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. Oct 16 01:07:54.429835 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '391edabf-85a9-4565-92f6-ebd0664d3e20'} Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:54.431076 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. Oct 16 01:07:54.434050 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.436006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Returning 404 to user: Instance 391edabf-85a9-4565-92f6-ebd0664d3e20 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:54.436006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-319cdd19-f4d9-456f-a3e5-be622757d9f2 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20" status: 404 len: 111 microversion: 2.1 time: 0.032196 Oct 16 01:07:54.436006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 516/1053] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:54 2022] GET /compute/v2.1/servers/391edabf-85a9-4565-92f6-ebd0664d3e20 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:54.934270 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:54.935005 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:54.941558 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.942052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.942492 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.943258 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.943971 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.944404 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.959104 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] HTTP exception thrown: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Request method failure captured: Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: DELETE /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab HTTP/1.1 Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1eG2j1-Qx4qQKlflgNnJ9o38mFemkYM9ej90vtQx4uBAfNoUgZe7Ad3_Gzmuz7XTwYgWUxXLq9IS5Op3iLbi9EUTnQj-YCg4Dc7hD7cRcPJpMaZq-g3E1x85xxWVY1RJ1q7Y9J8JNMmZVQ3NEpMJG2ruwTnOIZZdu0eVzhJOk8c_1c Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:07:54.960834 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cb1743385ea14cc39244c83e8fc88e7e", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cb1743385ea14cc39244c83e8fc88e7e"}]}] Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: c4e65cef89af48a7a51e5548634de75f Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:54.962529 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'e232da69-f419-4e6d-802e-a253ad2587ab'} Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:54.964061 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:54.965712 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:54.967373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Returning 404 to user: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:54.967373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-58ca8a9a-70a3-4a4d-94f1-361c59f67457 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "DELETE /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 404 len: 111 microversion: 2.1 time: 0.033114 Oct 16 01:07:54.967373 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 538/1054] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:07:54 2022] DELETE /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:54.972397 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:54.972960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] [instance: e232da69-f419-4e6d-802e-a253ad2587ab] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:54.980257 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.980745 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.981179 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.981889 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:54.982339 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:54.982774 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:54.998744 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] HTTP exception thrown: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Request method failure captured: Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab HTTP/1.1 Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1eG2j1-Qx4qQKlflgNnJ9o38mFemkYM9ej90vtQx4uBAfNoUgZe7Ad3_Gzmuz7XTwYgWUxXLq9IS5Op3iLbi9EUTnQj-YCg4Dc7hD7cRcPJpMaZq-g3E1x85xxWVY1RJ1q7Y9J8JNMmZVQ3NEpMJG2ruwTnOIZZdu0eVzhJOk8c_1c Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:07:55.000310 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/cb1743385ea14cc39244c83e8fc88e7e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_cb1743385ea14cc39244c83e8fc88e7e", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/cb1743385ea14cc39244c83e8fc88e7e"}]}] Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: cb1743385ea14cc39244c83e8fc88e7e Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-TestVolumeBootPattern-1252253557 Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: c4e65cef89af48a7a51e5548634de75f Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-TestVolumeBootPattern-1252253557-project Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:55.002006 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'e232da69-f419-4e6d-802e-a253ad2587ab'} Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:55.003128 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. Oct 16 01:07:55.004798 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.006299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Returning 404 to user: Instance e232da69-f419-4e6d-802e-a253ad2587ab could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:55.006299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6d95d460-99f8-4251-94ce-1ebdf93b8542 tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab" status: 404 len: 111 microversion: 2.1 time: 0.033375 Oct 16 01:07:55.006299 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 517/1055] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:54 2022] GET /compute/v2.1/servers/e232da69-f419-4e6d-802e-a253ad2587ab => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:55.243869 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:55.244736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] [instance: 38ce2d6c-6685-47ba-954e-94c647f15f86] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:07:55.252057 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:55.252546 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:55.252990 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:55.253692 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:07:55.254150 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:07:55.254578 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:07:55.270115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] HTTP exception thrown: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Request method failure captured: Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 HTTP/1.1 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1kD04V35otRakb9Ppzy-1tN67lkqTeUMwDUho28nP5g8wHex5wwygbyKP9Dl_i2_iFdSz7atLLxGTDK_kfHZOANFoHflkZsIKO9YklQxfQQX5CUvwkNfrTWLCPAftJxd4MIf43oRTvKBSHCe2shmIf5LX6eDE3bOVE_Af5pN2gIDm0 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: c322a47bb2364c8ea9250a7ecea537bf Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:07:55.271736 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c322a47bb2364c8ea9250a7ecea537bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c322a47bb2364c8ea9250a7ecea537bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c322a47bb2364c8ea9250a7ecea537bf", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c322a47bb2364c8ea9250a7ecea537bf"}]}] Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: c322a47bb2364c8ea9250a7ecea537bf Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-799418870 Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-VolumesSnapshotTestJSON-799418870-project Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: 602ff9dfc79d43839abbb179cbf6f9fd Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-VolumesSnapshotTestJSON-799418870-project Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. Oct 16 01:07:55.274390 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '38ce2d6c-6685-47ba-954e-94c647f15f86'} Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:07:55.276572 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. Oct 16 01:07:55.279451 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:07:55.282045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] Returning 404 to user: Instance 38ce2d6c-6685-47ba-954e-94c647f15f86 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:07:55.282045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-18a97500-765c-4f8e-b307-2ee34a95c489 tempest-VolumesSnapshotTestJSON-799418870 tempest-VolumesSnapshotTestJSON-799418870-project] 10.222.0.25 "GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86" status: 404 len: 111 microversion: 2.1 time: 0.033914 Oct 16 01:07:55.282045 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 539/1056] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:07:55 2022] GET /compute/v2.1/servers/38ce2d6c-6685-47ba-954e-94c647f15f86 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:07:56.312746 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9ba6a842-3add-4fbd-8fac-440a4ee5d20d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] Calling method '.version_select at 0x7f8bbdb03160>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:07:56.333960 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9ba6a842-3add-4fbd-8fac-440a4ee5d20d tempest-TestVolumeBootPattern-1252253557 tempest-TestVolumeBootPattern-1252253557-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-466177183" status: 202 len: 0 microversion: 2.1 time: 0.025075 Oct 16 01:07:56.334560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 518/1057] 10.222.0.25 () {58 vars in 1260 bytes} [Sun Oct 16 01:07:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-466177183 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 16 01:08:01.245900 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:01.248883 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:01.255959 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.256745 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.257185 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.257993 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.258374 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.258793 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.292679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-622672ea-02dc-4ab6-ac5e-384ab7f84913 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.050992 Oct 16 01:08:01.293144 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 540/1058] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:08:01 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 371 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:01.463037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:01.463770 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:01.485038 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.485558 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.488357 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.491476 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.492654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.493098 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.547131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-26f4a7ec-f36f-4c00-b4bc-e5039cacdff2 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd" status: 200 len: 194 microversion: 2.1 time: 0.089682 Oct 16 01:08:01.553157 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 519/1059] 10.222.0.25 () {58 vars in 1413 bytes} [Sun Oct 16 01:08:01 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd => generated 194 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:01.561469 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:01.562084 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:01.585874 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.586369 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.589958 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.592298 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:01.599454 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:01.601497 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:01.724971 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lazy-loading 'services' on Instance uuid 78ed622b-285f-4f9b-a373-bdfc9b667fc6 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:08:01.832179 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c8e9174e-5755-4656-abf0-e80980642f25 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd" status: 202 len: 0 microversion: 2.1 time: 0.275815 Oct 16 01:08:01.833158 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 541/1060] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:08:01 2022] DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/aaeb0149-234d-4fc6-9f49-732bc4872efd => generated 0 bytes in 279 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:08:03.007293 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:03.009454 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:03.029173 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:03.031869 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:03.036608 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:03.039192 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:03.039871 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:03.041996 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:03.097417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-9e07a4c6-2a66-4623-9bc0-5ae886749261 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/a43c05f3-8f7c-4226-9d8d-014c11f25285" status: 200 len: 194 microversion: 2.1 time: 0.094365 Oct 16 01:08:03.098352 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 520/1061] 10.222.0.25 () {58 vars in 1413 bytes} [Sun Oct 16 01:08:03 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/a43c05f3-8f7c-4226-9d8d-014c11f25285 => generated 194 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:03.112415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:03.113103 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:03.121472 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:03.122100 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:03.122618 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:03.123563 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:03.124259 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:03.124695 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:03.272761 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lazy-loading 'services' on Instance uuid 78ed622b-285f-4f9b-a373-bdfc9b667fc6 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:08:03.375279 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-9a740732-8b28-4362-906f-58ce10c99672 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/a43c05f3-8f7c-4226-9d8d-014c11f25285" status: 202 len: 0 microversion: 2.1 time: 0.269272 Oct 16 01:08:03.376017 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 542/1062] 10.222.0.25 () {58 vars in 1416 bytes} [Sun Oct 16 01:08:03 2022] DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments/a43c05f3-8f7c-4226-9d8d-014c11f25285 => generated 0 bytes in 271 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:08:04.600276 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:04.602426 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:04.610937 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.611759 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.612303 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.613089 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.613658 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.614089 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.645462 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-cee70aa3-7be4-4bd8-8ed1-f970b8ef084f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048999 Oct 16 01:08:04.646037 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 521/1063] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:08:04 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:08:04.774145 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:04.778052 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:04.784720 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.785214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.785785 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.786500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.786985 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.787415 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.818887 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-f937337a-110c-4e24-9e8d-c2e0d4bcac12 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048764 Oct 16 01:08:04.819464 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 543/1064] 10.222.0.25 () {58 vars in 1302 bytes} [Sun Oct 16 01:08:04 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 16 01:08:04.885039 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:04.885649 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:04.892260 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.892861 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.893370 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.894171 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:04.894700 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:04.895219 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:04.924245 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Going to try to terminate instance {{(pid=119013) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 16 01:08:05.012131 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-f52baf72-d227-4267-b8c9-604b34c1bb80 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 204 len: 0 microversion: 2.1 time: 0.131062 Oct 16 01:08:05.012946 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 522/1065] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:08:04 2022] DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 16 01:08:05.021190 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:05.021960 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:05.029408 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:05.029994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:05.030436 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:05.031171 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:05.031630 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:05.032055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:05.169099 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:05.177919 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-a0290510-7a3b-4bb2-8700-671d53cf77c4 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1673 microversion: 2.1 time: 0.160626 Oct 16 01:08:05.178505 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 544/1066] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:05 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1673 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:06.203436 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:06.204139 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:06.210692 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:06.211214 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:06.211654 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:06.212933 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:06.213632 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:06.214067 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:06.344951 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:06.353347 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-a2675935-a633-448d-bdd1-913c9fb9bead tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1673 microversion: 2.1 time: 0.153815 Oct 16 01:08:06.354230 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 523/1067] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:06 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:07.597362 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:07.598105 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:07.604083 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:07.604556 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:07.604988 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:07.605665 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:07.606115 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:07.606533 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:07.621679 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG dbcounter [-] [119012] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=4 {{(pid=119012) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:08:07.738689 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:07.747789 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-314ddce5-7d3d-4fcf-9ab2-84f62615f299 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1673 microversion: 2.1 time: 0.373777 Oct 16 01:08:07.748368 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 545/1068] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:07 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1673 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:08.773544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:08.774432 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:08.782903 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:08.783553 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:08.784017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:08.784979 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:08.785457 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:08.786087 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:08.805105 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG dbcounter [-] [119013] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=119013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 16 01:08:08.933998 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.policy [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119013) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:08.942329 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-4215bfef-feb8-429f-9006-34041b2a074d tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1673 microversion: 2.1 time: 0.173702 Oct 16 01:08:08.942947 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 524/1069] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:08 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1673 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:09.967780 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:09.968337 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:09.974786 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:09.975305 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:09.975749 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:09.976447 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:09.976899 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:09.977324 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:10.056015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:10.064199 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-36562b38-099a-4c3c-91e9-1581dac6cc8b tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1606 microversion: 2.1 time: 0.100289 Oct 16 01:08:10.064934 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 546/1070] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:09 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1606 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:10.666229 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "78ed622b-285f-4f9b-a373-bdfc9b667fc6", "name": "network-vif-deleted", "tag": "b75f86c7-a0ac-43e2-8da9-10f736efe195"}]} {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 16 01:08:10.676646 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:10.678082 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:10.678644 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:10.691442 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.compute.server_external_events [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] Creating event network-vif-deleted:b75f86c7-a0ac-43e2-8da9-10f736efe195 for instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 on hv-cinder-maste Oct 16 01:08:10.697271 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [req-5055fde3-6fc7-4c55-8fec-fde07a84a475 req-38d797cc-c935-46fb-8370-5e914897748c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104705 Oct 16 01:08:10.698234 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 525/1071] 10.222.0.25 () {64 vars in 1317 bytes} [Sun Oct 16 01:08:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:11.088951 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:11.089755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:11.096214 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:11.096709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:11.097153 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:11.097848 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:11.098311 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:11.098747 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:11.131709 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:11.132209 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:11.132750 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:11.133205 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.objects.instance [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lazy-loading 'fault' on Instance uuid 78ed622b-285f-4f9b-a373-bdfc9b667fc6 {{(pid=119012) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 16 01:08:11.181994 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.policy [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1283100ac2c439cb308b9c57ae31f40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8852bfe31aa4ba68190661d26c177cb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=119012) authorize /opt/stack/nova/nova/policy.py:199}} Oct 16 01:08:11.190460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-b9297107-3758-4ab5-a08f-0d0bc52e6ed7 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 200 len: 1327 microversion: 2.1 time: 0.105523 Oct 16 01:08:11.206649 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 547/1072] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:11 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 1327 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:12.214935 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:12.215638 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:12.221887 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:12.222380 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:12.222827 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:12.223560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:12.224025 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:12.224456 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:12.238102 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 HTTP/1.1 Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:08:12.239814 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:12.241500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '78ed622b-285f-4f9b-a373-bdfc9b667fc6'} Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:08:12.242428 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:12.244635 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:12.246034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:08:12.246034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2364d8bf-0fe2-4f5a-ab34-65651738460f tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 404 len: 111 microversion: 2.1 time: 0.031439 Oct 16 01:08:12.246034 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 526/1073] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:12 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:08:12.251098 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-c0d940de-2e56-4d5c-b786-54f08e79d350 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '.version_select at 0x7f8bbdb53160>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:12.270498 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-c0d940de-2e56-4d5c-b786-54f08e79d350 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2091634348" status: 202 len: 0 microversion: 2.1 time: 0.023187 Oct 16 01:08:12.271113 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 548/1074] 10.222.0.25 () {58 vars in 1221 bytes} [Sun Oct 16 01:08:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2091634348 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 16 01:08:15.157956 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:15.158505 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:15.165717 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:15.166458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:15.167100 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:15.168044 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:15.168560 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:15.168985 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:15.183801 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 HTTP/1.1 Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:08:15.185395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.187145 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': '78ed622b-285f-4f9b-a373-bdfc9b667fc6'} Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:08:15.188017 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.189417 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.190892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:08:15.190892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-bfd31d4a-5798-45b8-92ee-ecdbe91ca020 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 404 len: 111 microversion: 2.1 time: 0.032915 Oct 16 01:08:15.190892 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 527/1075] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:08:15 2022] DELETE /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:08:15.196149 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:15.196755 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: 78ed622b-285f-4f9b-a373-bdfc9b667fc6] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:15.202737 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:15.203263 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:15.203703 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:15.204394 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:15.204852 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:15.205281 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:15.219540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 HTTP/1.1 Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:08:15.221198 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.223055 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': '78ed622b-285f-4f9b-a373-bdfc9b667fc6'} Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:08:15.224015 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. Oct 16 01:08:15.225552 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:15.226969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance 78ed622b-285f-4f9b-a373-bdfc9b667fc6 could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:08:15.226969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-7e59b336-8a6c-4867-923f-09563062c9a6 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6" status: 404 len: 111 microversion: 2.1 time: 0.031092 Oct 16 01:08:15.226969 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 549/1076] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:15 2022] GET /compute/v2.1/servers/78ed622b-285f-4f9b-a373-bdfc9b667fc6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:08:16.363955 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:16.364549 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.compute.api [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119013) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:16.371767 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:16.372261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:16.372698 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:16.373458 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:16.373916 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:16.374345 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG oslo_concurrency.lockutils [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119013) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:16.387854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.wsgi [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: request: DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a HTTP/1.1 Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept: application/json Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Accept-Encoding: identity Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Connection: close Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Length: 0 Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Content-Type: application/json Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Host: 10.222.0.25 Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: Openstack-System-Scope: None Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: User-Agent: python-urllib3/1.26.12 Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Id: None Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Domain-Name: None Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Identity-Status: Confirmed Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Is-Admin-Project: True Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Id: default Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Domain-Name: Default Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Role: member,reader Oct 16 01:08:16.389414 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Roles: member,reader Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Id: default Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Domain-Name: Default Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: method: > Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: exception: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.391193 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: action_args: {'id': 'f3182305-f754-4b0f-989d-7362967cb10a'} Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:08:16.392306 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.394016 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.395895 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. {{(pid=119013) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:08:16.395895 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-6ec7bb8a-af52-4191-a2fc-f7662db5b8f8 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 404 len: 111 microversion: 2.1 time: 0.030676 Oct 16 01:08:16.395895 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 528/1077] 10.222.0.25 () {58 vars in 1239 bytes} [Sun Oct 16 01:08:16 2022] DELETE /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 16 01:08:16.399401 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Calling method '>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:16.400259 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.compute.api [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] [instance: f3182305-f754-4b0f-989d-7362967cb10a] Fetching instance by UUID {{(pid=119012) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 16 01:08:16.406928 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:16.407423 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:16.407862 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:16.408567 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Acquiring lock "d19e9fc7-2682-4022-903d-afef81878fc4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 16 01:08:16.409030 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 16 01:08:16.409460 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG oslo_concurrency.lockutils [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Lock "d19e9fc7-2682-4022-903d-afef81878fc4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=119012) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 16 01:08:16.423760 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.wsgi [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] HTTP exception thrown: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Request method failure captured: Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: request: GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a HTTP/1.1 Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept: application/json Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Accept-Encoding: identity Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Connection: close Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Length: 0 Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Content-Type: application/json Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Host: 10.222.0.25 Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: Openstack-System-Scope: None Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: User-Agent: python-urllib3/1.26.12 Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Auth-Token: gAAAAABjS1hKq-YyAwJ-aL002JuCjEPP_zli5XIOpQ-uptxNCj2tTXB9mPOithSvSCwVh_ZVtKNt7lM2MC1MmsjaAFLLj95E8Q_MwM9XdF-hDo_YrhXKc2bJVCOytiO189OB0JkY8vSNeQi5XQvUcTJaZdMm6U2KYrLf2z7GpkK9YVLG7LgFiqU Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Id: None Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Domain-Name: None Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Identity-Status: Confirmed Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Is-Admin-Project: True Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Id: default Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Domain-Name: Default Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Project-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Role: member,reader Oct 16 01:08:16.425308 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Roles: member,reader Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b8852bfe31aa4ba68190661d26c177cb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b8852bfe31aa4ba68190661d26c177cb", "adminURL": "http://10.222.0.25:8080"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b8852bfe31aa4ba68190661d26c177cb"}]}] Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Id: b8852bfe31aa4ba68190661d26c177cb Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-Tenant-Name: tempest-AttachVolumeTestJSON-860372929 Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Id: default Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Domain-Name: Default Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Id: e1283100ac2c439cb308b9c57ae31f40 Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: X-User-Name: tempest-AttachVolumeTestJSON-860372929-project Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: method: > Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: exception: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.426744 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: action_args: {'id': 'f3182305-f754-4b0f-989d-7362967cb10a'} Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 16 01:08:16.427605 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. Oct 16 01:08:16.429001 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: ERROR nova.api.openstack.wsgi Oct 16 01:08:16.431213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] Returning 404 to user: Instance f3182305-f754-4b0f-989d-7362967cb10a could not be found. {{(pid=119012) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 16 01:08:16.431213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8846760e-bf5b-40d3-8ba8-d09a6ffec707 tempest-AttachVolumeTestJSON-860372929 tempest-AttachVolumeTestJSON-860372929-project] 10.222.0.25 "GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a" status: 404 len: 111 microversion: 2.1 time: 0.031305 Oct 16 01:08:16.431213 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 550/1078] 10.222.0.25 () {58 vars in 1236 bytes} [Sun Oct 16 01:08:16 2022] GET /compute/v2.1/servers/f3182305-f754-4b0f-989d-7362967cb10a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 16 01:08:22.222003 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-afe13869-24d0-40ed-b749-619433436970 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb3f430>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:22.272261 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-afe13869-24d0-40ed-b749-619433436970 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/08469944-bd9e-4af1-a4d1-148b10613add" status: 200 len: 351 microversion: 2.1 time: 0.054031 Oct 16 01:08:22.272854 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 529/1079] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:08:22 2022] GET /compute/v2.1/os-volumes/08469944-bd9e-4af1-a4d1-148b10613add => generated 351 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:23.606540 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8ceec2db-6894-4232-9a52-483abfee9190 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb3e1f0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:23.655981 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8ceec2db-6894-4232-9a52-483abfee9190 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/a3578816-a049-4588-b713-aea820609acf" status: 200 len: 349 microversion: 2.1 time: 0.053257 Oct 16 01:08:23.656565 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 551/1080] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:08:23 2022] GET /compute/v2.1/os-volumes/a3578816-a049-4588-b713-aea820609acf => generated 349 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.000395 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-5a8eeaca-3394-494c-977e-17b2a74c98d2 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdaacee0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.048544 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-5a8eeaca-3394-494c-977e-17b2a74c98d2 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/7e79887e-ac8c-49ac-8a0c-f69f9766c908" status: 200 len: 351 microversion: 2.1 time: 0.052391 Oct 16 01:08:25.049126 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 530/1081] 10.222.0.25 () {58 vars in 1245 bytes} [Sun Oct 16 01:08:24 2022] GET /compute/v2.1/os-volumes/7e79887e-ac8c-49ac-8a0c-f69f9766c908 => generated 351 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.063328 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-117d47b0-7664-42d1-b5e0-a07a0dfd1a56 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbddc48b0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.147128 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-117d47b0-7664-42d1-b5e0-a07a0dfd1a56 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.088156 Oct 16 01:08:25.147702 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 552/1082] 10.222.0.25 () {58 vars in 1134 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:25.161855 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-be91cca1-621c-44cf-8e71-c47092d3fd9b tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdaacf70>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.248532 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-be91cca1-621c-44cf-8e71-c47092d3fd9b tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.090363 Oct 16 01:08:25.249108 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 531/1083] 10.222.0.25 () {58 vars in 1149 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.263624 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-51d8d962-d1e6-4751-bd79-798d85fca232 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb53e50>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.344121 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-51d8d962-d1e6-4751-bd79-798d85fca232 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.084226 Oct 16 01:08:25.344710 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 553/1084] 10.222.0.25 () {58 vars in 1134 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:25.358249 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-139933b8-8841-434e-8f4e-613e08f343d6 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb3f8b0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.440362 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-139933b8-8841-434e-8f4e-613e08f343d6 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 352 microversion: 2.1 time: 0.085850 Oct 16 01:08:25.441008 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 532/1085] 10.222.0.25 () {58 vars in 1167 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 352 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.454684 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-8959421d-57f2-4dfb-bcc3-ef565ed07473 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb7d4c0>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.538310 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-8959421d-57f2-4dfb-bcc3-ef565ed07473 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.087307 Oct 16 01:08:25.538992 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 554/1086] 10.222.0.25 () {58 vars in 1170 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.553247 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-2c602fba-56cf-4328-9050-c0aafb90ae47 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb1bca0>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.636500 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-2c602fba-56cf-4328-9050-c0aafb90ae47 tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.086892 Oct 16 01:08:25.637195 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 533/1087] 10.222.0.25 () {58 vars in 1155 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 16 01:08:25.651344 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: DEBUG nova.api.openstack.wsgi [None req-2ed1efec-0702-4339-8f06-fadf6d3ad03b tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb5d700>' {{(pid=119012) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.733922 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: INFO nova.api.openstack.requestlog [None req-2ed1efec-0702-4339-8f06-fadf6d3ad03b tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 352 microversion: 2.1 time: 0.086981 Oct 16 01:08:25.734500 dv-cinder-master-3-iscsi devstack@n-api.service[119012]: [pid: 119012|app: 0|req: 555/1088] 10.222.0.25 () {58 vars in 1188 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 352 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 16 01:08:25.748136 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: DEBUG nova.api.openstack.wsgi [None req-13d39c54-9df2-4e7d-b88f-e14abc01e95a tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] Calling method '.version_select at 0x7f8bbdb1b160>' {{(pid=119013) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 16 01:08:25.830987 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: INFO nova.api.openstack.requestlog [None req-13d39c54-9df2-4e7d-b88f-e14abc01e95a tempest-VolumesTestJSON-1275885966 tempest-VolumesTestJSON-1275885966-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.086531 Oct 16 01:08:25.831623 dv-cinder-master-3-iscsi devstack@n-api.service[119013]: [pid: 119013|app: 0|req: 534/1089] 10.222.0.25 () {58 vars in 1155 bytes} [Sun Oct 16 01:08:25 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0)